Need to build an interactive data pipeline monitoring dashboard which will show the status of each pipeline based on the logs data. Please find the attached mockup. Also, open for new dashboard designs. The goal is to upload this dashboard to Tableau Online or Server from where the daily report can be emailed out automatically.
This is how the logs look like -
id = id of a log
run_id = Pipeline Execution ID
audit_createdate = The day and time log file was created. This log was created immidiatly after pipeline was run.
datasource = This indicates the higher level data source
dataset = Under the data source there are multiple datasets. The logs are captured for each dataset.
record_count = This is number of rows that pipeline outputed into the tables. -1 indicates the execution was failed.
execution_start_time = start of the pipeline execution
execution_end_time = end of the pipeline execution
total_execution_time = Total time taken by the pipeline execution. format = hh:mm:[login to view URL]
execution_status = For datasource="business-data", the value is comma separated. First value is for Drop table and 2nd value is for Create table. Example: SUCCEEDED, SUCCEEDED ([login to view URL] table= SUCCEEDED,Create table= SUCCEEDED)or SUCCEEDED,FAILED (i.e. Drop table= SUCCEEDED,Create table= FAILED)
24 фрилансеров(-а) в среднем готовы выполнить эту работу за $163
I’ve been currently working on banks user data to build an automated dashboard and I’m convince your data will be similar to it and my expertise and experience will help me complete the work with more precision.
Hi, I am a Data Analyst and have strong hands on experience and Knowledge on visualization tools like Tableau, Looker. Have made many dynamic dashboards.