Founding Engineer, RL Environments (Equity-Only)
Founding Engineer, RL Environments Thankly · Remote · Full-time
About Thankly
Thankly is an AI commerce platform based in Brooklyn and backed by Plug and Play. We build branded shopping and checkout apps for retailers and brands on ChatGPT, Claude, Gemini, and other agent surfaces. Our platform decomposes a shopper’s request into product needs, retrieves matches from the retailer’s live catalog, and validates the results before it recommends anything.
About the role
We are building reinforcement learning environments for shopping agents. Most agent environments today rely on simulated users and hired experts. Ours start from real shopper questions and retail merchandising expertise, so they reflect what people actually ask and what a good answer looks like in commerce.
You will design and build these environments with our CTO and founding team. This includes the task sets, the rubrics, the autograders, and the benchmarks we use to train and evaluate agents. You will own the work from the first prototype to a system that partners can use.
What you’ll do
Design environment specifications for multi-turn shopping tasks, including the state, the actions an agent can take, and what counts as task success. Build task datasets from real shopper queries and product catalogs, with clear criteria for coverage and difficulty. Write rubrics and build autograders that score agent responses for product accuracy, relevance, catalog grounding, and policy compliance. Build LLM-as-judge graders and test how well they agree with human ratings. Run baseline evaluations across models from OpenAI, Anthropic, and Google, and report the results. Build the tooling to run environments at scale, track results, and catch reward hacking. Write documentation so that external researchers and partners can use the environments.
What we’re looking for
A degree or current study in computer science, machine learning, statistics, or a related field. Strong Python skills and experience with at least one ML framework, such as PyTorch or JAX. Coursework, research, or project experience in reinforcement learning, LLM evaluation, or agent systems. Experience working with LLM APIs and building prompts, tools, or evaluation pipelines. Comfort with ambiguity. You can define a problem, propose an approach, and ship a first version without a detailed spec. Clear written communication.
Nice to have
Familiarity with RL environment frameworks such as Gymnasium, or with agent benchmarks such as WebShop, τ-bench, or SWE-bench. Experience with RLHF, RLAIF, or reward modeling. Experience with embeddings, vector search, or recommendation systems. Interest in e-commerce or retail. Published research or open-source contributions in ML.
Why join
You will be one of the first engineers and will shape a new product line from the start. You will work directly with the founders on real retail data and real agent surfaces. Founding equity.