Advance Analytics and BI Platform

Current Use Case is to collect data from CRM service providers like SalesForce &

Microsoft CRM 365 and then transform into meaningful data by logically joining

different entities received and then persisting it into Data Warehouse.

Business Intelligence Dashboards will be developed and integrate it with Data

Warehouse and perform analysis using analytical queries (joining, grouping, sorting


Along with this, the Advanced Analytics Platform will be developed in which Data Science

The team will perform basic analysis first and Then build & train their Machine Learning

Models on top of it for Predictive Analytics & Recommendation Engines.

Create below modules -

1. Data Sources Management

Using this Module, User will be able to configure their Data Sources from which Data

needs to be collected.

Once User configure the Data Sources, Data Ingestion Job will be submitted to Apache

Gobblin and Gobblin will start collecting data from Data Sources.

2. Real-Time Analytics

Once Ingested Data is available on Kafka Streams, Structured Spark Streaming will be

used to process & transform it in a distributed way and write that to MariaDB Column


3. Data Lake

Data Lake will be required as Lot of Data will be ingested and Data Warehouse will be

having cleaned & transformed version of data .

All Data collected from CRM Data Sources will be coming in JSON format . So Apache

Gobblin will convert the json data into parquet format before loading in Data Lake for

better I/O & Low Latency Reads

4. Data Processing

For Data Processing, Apache Spark will be used which is distributed data processing

engine and Data Processing Jobs will be scheduled using Apache Airflow and it will

read latest data from Data Lake and apply required transformations and then persist the

data to Data Warehouse.

5. Data Warehousing

For Data Warehouse, Hive on Minio will be used and File Format will be Parquet. The hive

will act as a MetaStore and Schemas will be defined in it for various tables and Tables

will be pointing to their corresponding Minio Storage Location.

Business Intelligence

Both Querying Engines i.e. Spark sql on Hive and MariaDb ColumnStore supports JDBC .

So , Any BI Tool can connect to them using Standard JDBC Connections and execute

analytics queries and create various charts/graphs.

Квалификация: CRM, MySQL, PostgreSQL, Python, QlikView

Показать больше business intelligence platforms comparison, bi platforms gartner, enterprise analytics platform, best bi platforms, analytics platform architecture, business intelligence tools, best analytics platforms, magic quadrant for analytics and business intelligence platforms

О работодателе:
( 0 отзыв(-а, -ов) ) Reston, United States

ID проекта: #19178759

37 фрилансеров(-а) в среднем готовы выполнить эту работу за $4403


⭐⭐⭐⭐ i am Full stack developer. ⭐⭐⭐⭐⭐⭐ I read through the job details extremely carefully and I am absolutely sure that I can do the project very well. * 6+ years experience in web and mobile app development. * I Больше

$3000 USD за 30 дней(-я)
(73 отзывов(-а))

Howdy. How is your day? I just checked your project “Advance Analytics and BI Platform” and I have been similar project before. I am an experienced developer in CRM and HRM and I can handle your design to good. I Больше

$4705 USD за 30 дней(-я)
(47 отзывов(-а))

hy! Dear how are you? hope you will be fine. i read your project description and ready to work for you so please come in chat without wasting time so that we can start working quickly. i have good command in php, my Больше

$4941 USD за 30 дней(-я)
(22 отзывов(-а))

Dear Client, I have read your job post and existed to work with you n this project.I would like to discuss your project with you. Ping me when you are available for further discussion. I have already worked with Больше

$4444 USD за 30 дней(-я)
(17 отзывов(-а))

Hi! Nice to meet you! I have read your project description carefully. I am very interested in your project. I will offer you the best solution for your requirements. I am familiar with Machine learning and JDBC. Больше

$4000 USD за 30 дней(-я)
(23 отзывов(-а))

""Programming is my passion. I am well organized, can work along or within a team. I prefer small and medium projects, but large is also good"" please check my work below:- Wordpress: [login to view URL] Больше

$4444 USD за 30 дней(-я)
(52 отзывов(-а))

Hi. I am a Senior Software Engineer with rich experience in Machine Learning. I have very strong skills in Python, C/C++, C# and Java. I read the job description very carefully and understand what you want. I can p Больше

$3000 USD за 30 дней(-я)
(13 отзывов(-а))

Hi there, I have checked the details I have great experience with CRM, MySQL, PostgreSQL, Python, QlikView. Please start the chat so we can discuss this job more in detail. Thanks

$4500 USD за 30 дней(-я)
(10 отзывов(-а))

Hello How are you I have read your job description and it is very interesting to me I am familiar to python, mysql and postgresql I need more detail of your project Right now I have some unclear point in your proje Больше

$4444 USD за 30 дней(-я)
(13 отзывов(-а))

Hi there, I've gone through the post description carefully and ready to start work on this project. Requirements are very clear but please come over the chat so that i can get more requirements if remaining. I am Больше

$4444 USD за 30 дней(-я)
(12 отзывов(-а))

Hello Partner, SoluLab addresses the end-to-end challenges you face with end-to-end data engineering solutions that are more than just a pretty user interface. We automate most of the work for you, which is why our Больше

$12490 USD за 30 дней(-я)
(4 отзывов(-а))

I am in Chicago, USA. I would like to work on your project. I have more than 25 years of experience in web/software development and system administration. My latest work is [login to view URL] Have worked on Больше

$4941 USD за 30 дней(-я)
(8 отзывов(-а))

Dear sir, I am Tuan Tran in Vietnam, I have 12 year in Software Industry . I have experience in PHP (laravel, ci, yii ..)) , nodejs, angularjs, react (js and native), J2EE,and mobile applications. (android and iOS) My Больше

$4444 USD за 30 дней(-я)
(24 отзывов(-а))

My name is Li Ming. I have understood your project " Advance Analytics and BI Platform" , and am confidence on your project. We are team that consist of the young and enthusiastic 15 senior developers. There are mo Больше

$4444 USD за 30 дней(-я)
(2 отзывов(-а))

Strong in Java and Swift, website. It's Ming, the senior App and web developer. Once you check my reviews, you can see my skills and working quality. Have read the job description carefully with an exciting mo Больше

$4444 USD за 30 дней(-я)
(3 отзывов(-а))

Hi , We are really very much interested in working with you,we are also worked many BIG US companies like [login to view URL] [login to view URL] [login to view URL] [login to view URL] Больше

$4444 USD за 30 дней(-я)
(8 отзывов(-а))

Hello, I am pleasure with your job for Advance Analytics and BI Platform. Thank you for the job posting. It’s a pleasure to meet you. I’d really like to work with you on this one if possible! I do have a couple of Больше

$4444 USD за 30 дней(-я)
(14 отзывов(-а))

Hello Sir, I have great knowledge in core,cordignetor and Laravel. More than 6+ year experience.. I have read your requirement carefully. you need a CRM system right.?? I can do this job easily.. 1. Wh Больше

$3000 USD за 30 дней(-я)
(12 отзывов(-а))

Hi, I hope you are doing good. well regarding this project, I have extensive 8+ years of experience in Machine learning projects using Python. I will work with efficient and correct way. Give me work, I will surel Больше

$3000 USD за 7 дней(-я)
(29 отзывов(-а))

Hello, I have experience in big data management, i was working for telecom company and created BI tools from billing data for marketing, finance and management. We ware using QLICKVIEW engine for it. I will create Больше

$4444 USD за 30 дней(-я)
(38 отзывов(-а))