NLP Developer (Gen AI Evaluation Tools)
We usually respond within three days
🌟 Join Sigma.AI – Shaping the Future of Artificial Intelligence 🌍
🔹 What is Sigma?
Sigma is a leading global technology company specializing in data collection and annotation for Artificial Intelligence. With over 30 years of experience, offices in Spain, the US, and the UK, and operations in more than 200 languages, we support top multinational clients in developing cutting-edge AI solutions.
About the Job
We’re looking for a pragmatic, Python-focused engineer to join our R&D team supporting the evaluation of Generative AI systems. This role is responsible for the internal tools that power our annotation workflows, evaluation pipelines, dashboards, while streamlining key processes across the team. You'll help develop scalable, well-documented applications used across internal R&D as well as client-facing projects, contributing to papers and articles as needed.
You’ll work closely with linguists and project lead to design tools that are efficient, user-friendly, and robust enough to support both exploratory and production use cases. You should be comfortable rapidly prototyping internal demos, annotation pilots, or experimental evals, and just as capable of evolving those into maintainable, production-grade tools.
Required Qualifications
- 3+ years of experience programming in Python
- Experience building with LLM APIs and frameworks (e.g., OpenAI, Anthropic, Google, Langchain)
- Ability to transition from quick prototypes to robust, maintainable production code
- Experience with web frameworks (e.g., Flask or FastAPI) and basic frontend development (HTML, JS, Bootstrap)
- Strong familiarity with Linux and Bash scripting
- Experience managing and querying SQL databases (esp. SQLite)
- Experience with containerized development and Linux-based toolchains
- Comfortable handling structured data pipelines (e.g., JSONL, CSV, file systems)
- Familiarity with version control, reproducibility, and lightweight CI workflows
- Strong communication and collaboration skills across technical and non-technical teams
- Fluent in English
Preferred Qualifications
- Experience designing and implementing Agentic AI applications
- Familiarity with annotation tools (e.g., Label Studio, Doccano) and evaluation workflows
- Exposure to Hugging Face libraries, prompt templating, or model evaluation frameworks
- Basic understanding of NLP task structures and GenAI evaluation goals
- Experience building dashboards and visualizations (e.g., using Plotly, DataTables, or D3)
Salary: 80-90 K $US
- Department
- Annotation & Translation
- Locations
- United States
- Remote status
- Fully Remote
- Employment type
- Full-time
Sigma Ethics
Sigma follows a strong code of ethics upon which the company's culture is built. The principles drawn from this code guide all our professionals to perform quality work with integrity and accountability.
At Sigma, compliance with the law, maintaining a professional and independent stance, responsible decision-making, teamwork, continuous improvement and fluid communication with our clients are the pillars that allow us to be a world reference in the quality of the work we perform.
At Sigma, we ensure that everyone is treated fairly and equitably. We value professionalism, promote diversity and never allow any form of discrimination or any behavior that is not in line with the company's values and ethics. We promote dialogue and honest and respectful constructive criticism that drives us to constantly improve and value the work of others.
Our code of ethics helps us to fulfill our commitments to our clients, to generate an environment of trust and to maintain long-term relationships.
About Sigma Group
Help shape the future of ethical AI.
Learn more about Sigma.AI and Sigma Cognition.
Already working at Sigma Group?
Let’s recruit together and find your next colleague.