Закрыт

Code that will parse and merge large csv files

I'm working with large csv files up to 50GB in size. Typically one file is around 30GB while the one that will be merged onto it is just about 100 MB. The code I'm currently using will take about 1 hour to parse and merge two files. If we can reduce the time this takes by writing new code that will be great.

Навыки: Python, Программирование на C++

Показать больше: merge csv files with different columns python, merge two csv files python pandas, combine two csv files python, combine multiple csv files into one workbook separate sheets python, python merge multiple csv files, merging two csv files with a common column python, merge csv files python pandas, pandas merge csv files by column, graph large csv files, load large csv files mysql, load large csv files php, reading large csv files net, query large csv files, merge multiple csv files, merge horizontal csv files, query large csv files excel, google translate large csv files, how to open large csv files in excel 2010, free software to open large csv files, how to compare two large csv files

О работодателе:
( 0 отзыв(-а, -ов) ) Santa Clara, United States

ID проекта: #27364042

24 фрилансеров(-а) готовы выполнить эту работу в среднем за $176

liveexperts123

Hi there,I'm biddin on your project "Code that will parse and merge large csv files" I have read your project description and i'm confident i can do this project for you perfectly.I still have a few questions. please l Больше

$250 USD за 4 дней(-я)
(43 отзывов(-а))
7.0
hbxfnzwpf

I am very proficient in c/c++. I have 17 years c/c++ developing experience now, and have worked for more than 10 years. My work is online game developing, and mainly focus on server side, using c++ under Linux environm Больше

$100 USD за 1 день
(126 отзывов(-а))
6.8
sodiqa32

⭐⭐Hello, there!⭐⭐ ✅I have gone through your project details carefully and I think that I am the best fit developer for your project. ✅I have rich experience in similar projects, so I have a clear way to complete the pr Больше

$34 USD за 1 день
(25 отзывов(-а))
5.6
kirilnberezebko

Hi Ken T. I will parse the csv file. I am a senior python developer with 5+ years of experience. I have worked on several similar projects using selenium and flask and creating automation tool and can deliver quality r Больше

$200 USD за 2 дней(-я)
(18 отзывов(-а))
5.9
umg536

Hi there, I'm bidding on your project "Code that will parse and merge large csv files" Being an expert in C, and C++ programming I can do this project for you. please leave a message on my chat so we can discuss the b Больше

$250 USD за 4 дней(-я)
(10 отзывов(-а))
4.9
hasankassim7

Hello I am a Python and c++ expert. I have worked on a project of sorting very large files, so I am capable of doing yours. Let me know please about the criteria of merging. Feel free to send a message.

$80 USD за 2 дней(-я)
(29 отзывов(-а))
4.6
Shendiqerimi

Hello, Ken! I have read your description very carefully, so I'm very confident in your project. I have rich experiences in CSV parsing with python and done many projects such as your projects . In your pro Больше

$100 USD за 2 дней(-я)
(9 отзывов(-а))
4.4
casper70

Hi I am fully experienced working with python and c++. Contact me to discuss further details. Thanks for your time :)

$300 USD за 3 дней(-я)
(14 отзывов(-а))
4.8
igorshapo926

Hello. Thanks for your attention. I am Machine Learning, Deep Learning, Computer Vision expert and working in these fields from the past 5 years. The major field in which I have worked on including Deep Learning, Machi Больше

$150 USD за 1 день
(3 отзывов(-а))
4.0
CrazyProger

I'm a Python expert with rich experience in CSV. I read the job description and understood the requirements. I think I can finish this project within short period with a minimal budget. I can start work immediately. Больше

$140 USD за 2 дней(-я)
(6 отзывов(-а))
4.0
drozhkov

hi, probably i can help you 100chars100chars100chars100chars100chars100chars100chars100chars100chars

$250 USD за 7 дней(-я)
(7 отзывов(-а))
4.1
DRG1990

Hello I have read your job description carefully. Also, I have rich experience in python So I am interested in and I am confident in your project. I always provide perfect results to my clients as you can see my profil Больше

$140 USD за 3 дней(-я)
(9 отзывов(-а))
4.1
QASIM7862001

I believe i can reduce the time by alot get in touch Hey there, i am developer from the UK with over 9 years experience in web development. Upon reading your project description this seems like a task which i can start Больше

$250 USD за 1 день
(3 отзывов(-а))
3.8
Devesh23

Hello, I am kernel developer and I might have an idea what might be happening in your code. Your code is trying to read the file and work on it at runtime. Now, its a big file and if you are using C++ then you must be Больше

$250 USD за 7 дней(-я)
(12 отзывов(-а))
3.8
shahzain93

Hi, I can optimize code, Please share your current code so i can check and get back to you? what about language, can I use according to my choice? Thanks

$220 USD за 7 дней(-я)
(2 отзывов(-а))
3.1
Dutchstudent7750

Hello. So in order to speed up the merging there's a few things to few things to try, optimizing the query, choosing a different data structure and parallelization. I would need to take a look at what exactly it is yo Больше

$100 USD за 1 день
(9 отзывов(-а))
3.1
KMilan27

Hello, I can reduce the time using Multi Processing in Python. Let's discuss more in chat. Thank you Milan

$200 USD за 3 дней(-я)
(2 отзывов(-а))
2.5
Wellkuka

Hi I can do this job. There are several libraries in python for handling large data. I can start working right now.

$140 USD за 7 дней(-я)
(1 отзыв)
2.2
jainrakesh1701

I have 8 years experience in Python programming. Using python I have processed huge volume of data(in GB) and extracted meaningful information from it.I have worked with various formats like Json, Csv, Excel, XML, Par Больше

$200 USD за 7 дней(-я)
(0 отзывов(-а))
0.0
Krimson1991

Hi, In my previous job I did a lot of data analyzing with selfmade scripts in Python and C++. I think I can help you to create a better script for it. Can you tell me more about what kind of data needs to processed? Больше

$200 USD за 14 дней(-я)
(0 отзывов(-а))
0.0