Python LLM Performance Testing Expert -- 5
$15-25 USD / hour
I am in need of a Python expert who can perform a deep dive into the performance of various LLMs under different conditions. The main focus of this task is to benchmark different LLMs and evaluate their performance against each other.
Key responsibilities:
- Conducting performance tests on LLMs.
- Analysing the results and comparing them to identify the best performing system.
- Providing recommendations on how to optimize the performance of the chosen LLM.
- Set up a local testing environment (chatbot, playground dashboard, Jmeter 5.5) (done)
- Set up a cloud server (API server) (half done)
- Google Colab, Jupyter Notebook
- All configuration, test scripts in python, bash and exe)
- Dockerized images for future implementation
- Our Korean dataset of 20,000 published to HuggingFace workspace based on an existing Korean dataset
- documentation 1. testing items/procedures 2. instructions, manuals of this project.
You will also be required to integrate the API server with third-party APIs. Experience in this area is highly beneficial.
Ideal Skills:
- Strong expertise in Python programming.
- Previous experience in performance testing, specifically LLMs.
- Familiarity with benchmarking tools and methodologies.
- Ability to work with APIs and integrate them effectively.
Models to consider:
llama3-8b-8192
text-davinci-003
Mixtral 8x22b
Whisper
Reference:
[login to view URL]
Sample testing items (send me a permission request with your name plz)
[login to view URL]
Project term: 4 days
1. Performance Testing: I will execute comprehensive performance tests on multiple LLMs using Jmeter 5.5 to evaluate their efficiency under varied load scenarios. (also per my Googledoc sample and updated list of items)
2 Results Analysis: Utilizing tools like Google Colab and Jupyter Notebook, I will analyze and compare the performance metrics to identify the superior LLM. (I got a comparable metrics to compare, and you should match or excel )
by 18 hours
3. Optimization Recommendations: Based on the results, I will provide specific recommendations to enhance the chosen LLM’s performance. by 1 day
4. Local Testing Environment Setup: I will configure a local environment including a chatbot, dashboard, and necessary tools, all containerized with Docker for consistency. (dashboard like playground of Chatgpt; Jmeter with config scropts)
5. Cloud Server Configuration: I will establish and configure a cloud-based API server to handle and simulate real-world user requests.
6. Scripting and Automation: Development of all configuration and test scripts will be done in Python and Bash, with executable scripts as needed.
by 2nd day
7. Dockerization for Scalability: I will create Docker images of the testing setup to ensure easy deployment and future scalability. (both testing PC and end-server)
8. Dataset Publication on HuggingFace: Your Korean dataset of 20,000 entries will be published and integrated into the HuggingFace workspace, enhancing its accessibility and utility.
9. Documentation: I will produce two sets of documents: detailed testing procedures and user manuals for project replication and maintenance.
by 3rd and 4th day
progress reference link:
https://github.com/aiegoo/2024=04=[login to view URL]
Our datasets;
[login to view URL]
Project ID: #38034907
About the project
36 freelancers are bidding on average $24/hour for this job
Greetings, Can you provide more details about the specific load scenarios and performance metrics we should prioritize during testing? Are there any particular challenges or bottlenecks you've encountered in previous p More
Hello, Greetings! I have carefully checked your job post and am interested in working with you on this project, as it aligns with my skill set. Please take a look at my profile for confirmation. After talking about it More
I can enhance the performance and improve the latency easily. I did a similar project for many clients!
Hi, I have +7 years of experience dealing with machine learning algorithms and worked on multiple projects in this field, Please contact me to discuss more. Have a nice day
Hello After checking the job posting - Python LLM Performance Testing Expert -- 5, I felt that your project was similar to one I had worked on before. I have undertaken similar projects to ensure I can provide you wit More
Hi, ------SCRAPING, AUTOMATION EXPERT------- Please check my portfolio, I have scraped and created automation script for tons of websites. OpenAI, Python, Software Architecture, Docker and Software Testing are my core More
Hello, We are thrilled about the opportunity to collaborate on your project focused on benchmarking and optimizing various Language Models (LLMs). With our expertise in Python programming, performance testing, and API More
Hello Team, I'm Amit S., a seasoned Full Stack Developer with over 5 years of comprehensive experience. My expertise spans Python, JavaScript, Mobile App Development, CSS/HTML, Mean, Mern, and React.js. Technical Exp More
With strong expertise in Python programming and previous experience in LLM performance testing, we are ready to start right now. We have successfully conducted similar projects and can showcase our past projects for yo More
Hi, I can conduct a comprehensive performance evaluation of various Language Model Models (LLMs) under different conditions. I am confident in my ability to execute thorough performance tests using tools like Jmeter 5 More
Hello, I've thoroughly reviewed your job description, and I'm prepared to meet all your requirements. With over 5 years of experience in Python development and performance testing, I'm well-suited to benchmark various More
Hi, I hope this proposal finds you well. I have read the project description and I can do this project for this as I am an AI developer and ML expert and I have experience working with large language models, API develo More
I am a Python expert specializing in performance testing, with a keen interest in benchmarking LLMs. With extensive experience in Python, software testing, and Docker, I am well-equipped to conduct thorough performance More
