DevOps: Enable concurrency of Machine Learning pipelines (Kubernetes / AWS / GitLab runners)

MLReef is the collaboration platform for Machine Learning projects. It is an "end-to-end" development platform that relies heavily on cloud computing for running data pipelines.


The main objective of this task is to enable a per account concurrency in how many pipelines a user can execute at the same time. (e.g. As a user I want to start 10 experiments at the same time). This concurrency is mainly controlled by a. the user membership tier and b. the infrastructure of MLReef.

MLReef has built its services using the CE edition of GitLab. It includes GitLab runners that spawn machines for the data pipelines. These runners are limited in their capabilities (one runner can spawn and manage around 10 machines at the same time). We need a method for scaling GitLab runners based on real-time usage by the users and dependent on the machine type the user selected.

The position required here is DevOps focused (in close work with backend developer), who has experience in:

- AWS Services

- Kubernetes

- Ideally GitLab Runners (not mandatory, as these are "simple" tasks)

Навыки: Amazon Web Services, Java, Linux, Kubernetes, GitLab

О клиенте:
( 0 отзыв(-а, -ов) ) Neidling, Austria

ID проекта: #32187350

13 фрилансеров(-а) готовы выполнить эту работу в среднем за €27/час


Hi there,I'm biddin on your project "DevOps: Enable concurrency of Machine Learning pipelines (Kubernetes / AWS / GitLab runners)" I have read your project description and i'm an expert in Machine learning/Python/C++/J Больше

€44 EUR / час
(23 отзывов(-а))
(7 отзывов(-а))
(14 отзывов(-а))

Hi, We are AWS experts and certified and can do this. We can deploy your gitlab runner in either ec2 autoscalling or in kubernetes with cluster scaler . Please get in touch to start. Thanks

€33 EUR / час
(52 отзывов(-а))

Hello , I am a developer based out of Hyderabad , India . I have developed and hosted a few applications for my clients on PHP , Laravel , Nodejs , Vuejs , React and React Native . I have also helped my clients Больше

€20 EUR / час
(7 отзывов(-а))

☀️☀️☀️☀️☀️ Hello , I hope you are safe and Doing well I have seen your project requirements , I am looking to discuss further with you Hope we will meet soon to discuss further Больше

€27 EUR / час
(9 отзывов(-а))

Hi, I am a web designer & developer with 9+ years of experience. I have expertise in Laravel, CodeIgniter, Magento, WordPress, API, CMS, ecommerce, JavaScript, HTML, CSS, MySQL, PHP, Angular, React, React Native, Vue.j Больше

€22 EUR / час
(1 отзыв)

Hi, client. I am Highly interested in your job because I am living on DevOps having 3+ years experiences. Last year I have developed several Devops projects. I can show you those to prove my ability. please send me a m Больше

€27 EUR / час
(0 отзывов(-а))

Hello! I can surely help you with end-to-end development with my highest quality performance, having 2+ year of experience as a AWS, ML, Cloud Computing, Kubernetes ,Github, Jenkins, Docker, Linux, CI/CD, and more. Больше

€20 EUR / час
(0 отзывов(-а))

Hello, I am Koki based in Japan, Fullstack web developer with over 7 years of hands-on experience in web application development I have read the job description and found it very interesting and I am a good fit for th Больше

€30 EUR / час
(0 отзывов(-а))

We have expertise in Azure Architect and AWS, Azure & GCP Certified Professional Cloud Architect with multi-tasking skills having more over 10 years of extensive experience in AWS, Azure & Google Cloud Architecture, Cl Больше

€18 EUR / час
(0 отзывов(-а))

Our DevOps expertise: Our DevOps help in transitioning your delivery solutions to the cloud. We Build, test, and deploy applications on the DevOps Cloud with Cloud native tools. We can Dockerize your applications, wri Больше

€25 EUR / час
(0 отзывов(-а))

Hi, I have gone through your requirement, and I understood very well that you want Enable concurrency of Machine Learning pipelines. I have more than 4+ years of experience working as DevOps and I have manages more t Больше

€25 EUR / час
(0 отзывов(-а))