Multimodal Research Engineer - US Remote

Here at Hugging Face, we’re on a journey to advance good Machine Learning and make it more accessible. Along the way, we contribute to the development of technology for the better.

We have built the fastest-growing, open-source, library of pre-trained models in the world. With more than 1 Million+ models and 320K+ stars on GitHub, over 15.000 companies are using HF technology in production, including leading AI organizations such as Google, Elastic, Salesforce, Grammarly and NASA.

About the Role

We are in a “GPT-3 moment” for multimodal models: vision and language models are an order of magnitude more accurate and versatile than what they were 18 months ago, and yet still make relatively frequent mistakes. Meanwhile, having strong open-source and open-access alternatives to close-source models is critical for the healthiness and sustainability of the AI community, including AI research.

In this role, you will contribute to building and open-sourcing strong large-scale multimodal artifacts to support the community. That includes trained models, large-scale datasets (sourced and synthetically created), tools/libraries, and demos. 

Key responsibilities include:

• Creating synthetic data or sourcing specific data to fill identified gaps or address failure cases of vision and language models
• Brainstorming and iterating over data filtering pipelines
• Designing, building and maintaining evaluation pipelines to answer research questions
• Scaling and optimizing trainings on 100s of GPUs
• Expanding model capabilities with the latest training recipes (e.g. DPO) or expanding model modalities both on the input and output side (e.g. video, audio)
• Supporting and collaborating with external contributors and users to increase the impact of created artefacts 
  

About You

Your background can vary along a wide gradient. Perhaps you are familiar with large-scale challenges such as optimizing large-scale model training on 100s/1000s of GPUs. Or perhaps you are an expert in an area of ML such as computer vision, audio processing or NLP, and know the ins and outs of these systems including training and evaluation. Or perhaps you have scaled data and annotation pipelines to create large-scale datasets.

Most importantly, you are passionate about open science and open-source, and enjoy making ML more accessible to the larger community. You are autonomous, have strong engineering foundations, and are eager to learn about new areas of ML you are not familiar with. You care about building artifacts that people use and improve.

If you're interested in joining us, but don't tick every box above, we still encourage you to apply! We're building a diverse team whose skills, experiences, and background complement one another. We're happy to consider where you might be able to make the biggest impact.

More about Hugging Face

We are actively working to build a culture that values diversity, equity, and inclusivity. We are intentionally building a workplace where people feel respected and supported—regardless of who you are or where you come from. We believe this is foundational to building a great company and community. Hugging Face is an equal opportunity employer and we do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.

We value development. You will work with some of the smartest people in our industry. We are an organization that has a bias toward impact and is always challenging ourselves to continuously grow. We provide all employees with reimbursement for relevant conferences, training, and education.

We care about your well-being. We offer flexible working hours and remote options. We offer health, dental, and vision benefits for employees and their dependents. We also offer flexible parental leave and paid time off.

We support our employees wherever they are. While we have office spaces in NYC and Paris, we're very distributed and all remote employees have the opportunity to visit our offices. If needed, we'll also outfit your workstation to ensure you succeed.

We want our teammates to be shareholders. All employees have company equity as part of their compensation package. If we succeed in becoming a category-defining platform in machine learning and artificial intelligence, everyone enjoys the upside.

We support the community. We believe major scientific advancements are the result of collaboration across the field. Join a community supporting the ML/AI community.