blob: 4c8c5911e367626ee6a5abf8c818a3458c8016be (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
|
# Go-Druid-MySql-Influx
The objective of this cron script is to collect revenue related statistics (Views, Clicks, Leads) from MySql, and Druid datasources, and write those into InfluxDB so that we can eventually compare them - check for discrepancies.
Once the data is in InfluxDB, we can setup dashboards, alerting etc. from Granfana, using InfluxDB as its datasource.
## Pre-requisites
1. docker, docker-compose
2. If running without docker, then we need Go and a few dependencies:
- `github.com/go-sql-driver/mysql`
- `github.com/shunfei/godruid`
- `github.com/influxdata/influxdb1-client/v2`
## Build and Run Locally
> If running in Dev, uncomment the env_file (*.dev) in the docker-compose.yaml, and comment out the first one (prod).
- Build + Run: `docker-compose build --no-cache && docker-compose up --force-recreate`
- Only Run: `docker-compose up --force-recreate`
### Run tests
- Get the assert dependency: `go get -d github.com/stretchr/testify/assert`
- Run tests: `go test ./src/`
## Required Env Vars
If being run manually (docker-compose), then are set imported in the docker-compose file under "env_file".
If being run from k8s, then these are set in the k8s ConfigMap object.
```
JOB_NAME
TZ
WINDOW_SIZE
MYSQL_HOST
MYSQL_USER
MYSQL_PASSWORD
MYSQL_DB
DRUID_HOST
INFLUX_HOST
INFLUX_PORT
INFLUX_DB
INFLUX_RP
INFLUX_MEASUREMENT
DVO_API_CHECKIN_ENDPOINT
```
|
