Difference between revisions of "Automation Analytics"

From tannerjc wiki
Jump to: navigation, search
(How do I reset the gather time in tower?)
(How do I reset the gather time in tower?)
Line 277: Line 277:
; NOTE: the timestamp won't update if gathering is done from the awx-manage gather_analytics cli
; NOTE: the timestamp won't update if gathering is done from the awx-manage gather_analytics cli
awx-manage shell_plus
awx-manage shell_plus -c 'import awx.main.tasks as t; t.gather_analytics()'
import awx.main.tasks as t

Latest revision as of 13:25, 17 November 2020

frontend dev setup

use nvm to install and use node 10.15
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh | bash
source ~/.bashrc
nvm install 10.15
nvm use 10.15
clone the various repos
mkdir -p ~/workspace/github/RedHatInsights
git clone https://github.com/RedHatInsights/tower-analytics-frontend ~/workspace/github/RedHatInsights/tower-analytics-frontend
git clone https://github.com/RedHatInsights/insights-proxy ~/workspace/github/RedHatInsights/insights-proxy
fix the frontend spandx config
diff --git a/config/spandx.config.js b/config/spandx.config.js
index c40e554..611bc40 100644
--- a/config/spandx.config.js
+++ b/config/spandx.config.js
@@ -9,6 +9,5 @@ module.exports = {
         '/apps/automation-analytics': { host: `https://${localhost}:8002` },
         '/ansible/automation-analytics': { host: `https://${localhost}:8002` },
         '/beta/ansible/automation-analytics': { host: `https://${localhost}:8002` },
-        '/beta/config': { host: `http://${localhost}:8889` }
set the hosts file
cd ~/workspace/github/RedHatInsights/insights-proxy
sudo ./scripts/patch-etc-hosts.sh

start the proxy [requires docker or podman-docker]
cd ~/workspace/github/RedHatInsights/insights-proxy
SPANDX_CONFIG=~/workspace/github/RedHatInsights/tower-analytics-frontend/config/spandx.config.js ./scripts/run.sh

install the npm packages
cd ~/workspace/github/RedHatInsights/tower-analytics-frontend
npm install
DEBUG='express:*' npm start

open the "prod" url


fullstack dev setup

The backend stack is in a private repo https://github.com/RedHatInsights/tower-analytics-backend

The stack is heavily reliant on docker-compose to spin up and podman-compose will fail if used.

fix the spandx config
[jtanner@corsair tower-analytics-frontend]$ git diff config/local-api.spandx.config.js
diff --git a/config/local-api.spandx.config.js b/config/local-api.spandx.config.js
index 0645a59..4529b37 100644
--- a/config/local-api.spandx.config.js
+++ b/config/local-api.spandx.config.js
@@ -11,7 +11,6 @@ module.exports = {
         '/apps/automation-analytics': { host: `https://${localhost}:8002` },
         '/ansible/automation-analytics': { host: `https://${localhost}:8002` },
         '/beta/ansible/automation-analytics': { host: `https://${localhost}:8002` },
-        '/beta/config': { host: `http://${localhost}:8889` },
         '/api/tower-analytics': { host: `http://${localhost}:8000` }
run the insights proxy
cd ~/workspace/github/RedHatInsights/insights-proxy
SPANDX_CONFIG=~/workspace/github/RedHatInsights/tower-analytics-frontend/config/local-api.spandx.config.js ./scripts/run.sh
run the backend
cd ~/workspace/github/RedHatInsights/tower-analytics-backend
docker-compose -f local_ui_dev.yml kill
docker-compose -f local_ui_dev.yml down
docker image prune -a
sudo rm -rf integration_test/__pycache__
docker-compose -f local_ui_dev.yml up
migrate the database and make some fake data
cd ~/workspace/github/RedHatInsights/tower-analytics-backend
make init

fix the proxy container's hosts file
cd ~/workspace/github/RedHatInsights/tower-analytics-backend
docker-compose -f local.yml ps
docker exec -it <proxy-container-name> /bin/bash
echo " prod.foo.redhat.com" >> /etc/hosts
run the frontend
nvm use 10.15
cd ~/workspace/github/RedHatInsights/tower-analytics-frontend
npm install
DEBUG='express:*' npm start

open the urls ...

local insights.chrome

# checkout and build
git clone https://github.com/RedHatInsights/insights-chrome
cd insights-chrome
npm install
npm run build

# symlink apps/chrome to build
mkdir apps; cd apps
ln -s ../build apps/chrome
ln -s ../node_modules node_modules
ln -s chrome*.css chrome.css
cd js
ln -s chrome*.js chrome.js

# start httpd
npm install http-server
node_modules/http-server/bin/http-server -a -p 8080

Add the following entry to the spandx config ...

'/apps/chrome': { host: `http://${localhost}:8080` }

The insights-proxy has a builtin flag to do something similar to this ... https://github.com/RedHatInsights/insights-proxy/blob/a3a0b4bbf3ab4348ba893416dd2d85113cea5516/spandx.config.js#L138

compose backend and local frontend

  • commment out the frontend in local.yml
  • create the images with
    make build
  • create the backend stack with
    make up
    make init
  • use
    docker exec -it toweranalyticsbackend_insights_proxy_1 /bin/bash
    to edit the proxy container's /etc/hosts and set the frontend IP to the container's gateway
  • spin up the frontend with
    npm run start:container
    so that the service binds to instead of


How do I enable calls to the browser console in the frontend code?

eslint disabes console.log entries in the code, and needs configuration to allow it ...

diff --git a/.eslintrc.yml b/.eslintrc.yml
index c5d7d33..61835e5 100644
--- a/.eslintrc.yml
+++ b/.eslintrc.yml
@@ -72,6 +72,7 @@ rules:
   new-cap: 2
   no-bitwise: 2
   no-caller: 2
+  no-console: "off"
   no-mixed-spaces-and-tabs: 2
     - error
Another option is to add
/*eslint-disable */
at the top of the individual files.

How can I connect to the local backend postgres?

docker-compose -f local.yml exec postgres /bin/sh
su - postgres	

How can I see what active queries are running in postgres?

SELECT * FROM pg_stat_activity WHERE state = 'active';

How do I kill a query in postgres?

SELECT pg_terminate_backend(<pid>);

How do I run a single frontend test file?

node_modules/jest/bin/jest.js --coverage=false src/Charts/PieChart.test.js

How do I fix inotify for vscode?

[jtanner@jtx1 tower-analytics-frontend]$ cat /etc/sysctl.d/01-watches.conf 
fs.inotify.max_user_watches = 524200
[jtanner@jtx1 tower-analytics-frontend]$ sudo systemctl restart systemd-sysctl.service
[jtanner@jtx1 tower-analytics-frontend]$ cat /proc/sys/fs/inotify/max_user_watches

What should I do if the UI spins on a white page and never proceeds?

Check the network tab in chrome dev utils and look for any red entries. Double click on those and accept the certs.

How is a new nav item added to the leftnav?

https://github.com/RedHatInsights/cloud-services-config serves a route for "config/main.yml". This file contains a yaml'ized structure of the leftnav ...

  title: Automation Analytics
      - v1
  deployment_repo: https://github.com/RedHatInsights/tower-analytics-frontend-build.git
      - /ansible
      - /ansible/automation-analytics
      - id: clusters
        title: Clusters
        default: true
      - id: organization-statistics
        title: Organization Statistics
      - id: automation-calculator
        title: Automation calculator
      - id: notifications
        title: Notifications
      - id: stuff
        title: My STUFF
  source_repo: https://github.com/RedHatInsights/tower-analytics-frontend

How can I curl local backend urls?

curl -L  -H "x-rh-identity: $(echo '{"identity":{"internal":{"org_id":1}}}' | base64 -)" -v ; echo ""

How do I reset the gather time in tower?

curl -X PATCH -d '{"AUTOMATION_ANALYTICS_LAST_GATHER": null}' -H "Content-Type: application/json" -k -L -u admin:redhat1234 https://localhost/api/v2/settings/all/ | jq .AUTOMATION_ANALYTICS_LAST_GATHER
the timestamp won't update if gathering is done from the awx-manage gather_analytics cli
awx-manage shell_plus -c 'import awx.main.tasks as t; t.gather_analytics()'


podman permission or OCI errors ...
sudo rm -rf  ~/.local/share/containers/