BoodleBox
Senior Software Engineer — AI Systems & Scalability
Remote
·
Full time
Company website
You're a backend engineer who is at home in a Scala service and a Python service on the same day. You take it personally when a request is slow. You've stood up a model behind an API under real load, watched the GPU utilization, and can tell us what it cost per request. You have enough MLOps to run models in production and no interest in training them. You find the bottleneck before you ask for more machines. BoodleBox is already a generative AI platform in production: Scala and Python microservices on Kubernetes, hundreds of thousands of users, models served today through managed providers. We're not building an ML platform or running experiments. We want to make the one we have faster, more scalable, and more flexible about where models run. That includes evaluating and implementing self-hosted inference — embeddings, re-rankers, document-processing models, possibly open-weight LLMs — instead of relying only on providers like Azure AI Foundry, with everything that implies for latency, scaling, and cost. You'd sit in our AI Systems group alongside a retrieval specialist and the engineer growing into leading that group. You're the specialist on scalability and inference. You'll contribute across the AI and application backend where services meet, through missions with clear owners, without becoming the engineer everyone sends their hard problems to. This role is fully remote, with a company hub in Colorado Springs, CO.
About BoodleBox
BoodleBox is a VC-backed GenAI platform built for higher education. With 100,000+ users across 1,200+ colleges and universities, we partner with faculty, staff, and students to make responsible AI adoption practical, impactful, and lasting. Our platform brings together leading AI tools — including ChatGPT, Claude, Gemini, and Perplexity — under one roof, purpose-built for the unique demands of academic environments. We're guided by three core values: service, education, and collaboration.
Description
What You'll Own
- Production services in Scala and Python. The integrations, middleware, and services that connect generative AI to the application platform, built with monitoring and recovery from the start.
- Scalability and latency. Finding and removing bottlenecks in request handling, throughput, and concurrency so the platform handles more users at lower cost per request.
- Self-hosted inference. The evaluation, the serving architecture, GPU utilization, autoscaling, and the honest cost comparison against managed providers — then the implementation if the numbers support it.
- Practical MLOps. Models monitored in production, inference infrastructure versioned and operated reliably. No training pipelines; that's not what we do.
- Your services in production. A set of AI services you steward with a named backup: healthy, documented, safe for other engineers to change.
- Backend work inside missions. Cross-capability projects led by someone else where you own the AI-systems piece.
- The numbers. Load, cost per request, p99 latency — before and after, written down.
Who You Are
- You write production backend code in Python and Scala. Strong in Python, and either strong in Scala or demonstrably quick to get there on the JVM.
- You've diagnosed a distributed system under load. From logs, metrics, and traces, to the actual cause, not to “add replicas.”
- You've served a model behind an API. Concurrency, resource limits, the moment it fell over, what you did.
- You know the managed-versus-self-hosted trade. When it's worth running it yourself and when it isn't, in dollars and in pages.
- You own outcomes. Design through deploy through the 3am follow-through.
- You're comfortable being the specialist, not the lead. Someone else is growing into leading this group. You're here for depth.
- You've built something with an LLM. Not a wrapper. Something with real thought behind the architecture.
We're Not Checking Boxes. We're Looking for the Real Thing.
Didn't come up through a traditional path? Good. Tell us about it.
No hyperscaler logo on your resume? We've hired people who outperformed everyone who did.
Career gap? Weird pivot? Started somewhere completely left field? Bring it. The people who got here the strange way usually have the best instincts.
What we actually want to see: someone who finds the bottleneck before asking for more machines without thinking about it. If that's you, the path you took to get here is just a good story.
Who Thrives Here
We move quickly, and the speed comes from engineers who own their services end to end. You'll deploy your own code. You'll see a latency improvement land in a student's chat the same day. The platform team is small — one DevOps lead, one DevSecOps engineer — which means production stability is real engineering work here, not a ticket you file.
You'll fit if you like both halves of our stack, if cost-per-request is a number you'd check for fun, and if you want your scaling work to matter to a university that just rolled AI out to 30,000 students. We're building for education, in regulated environments. Reliability isn't a feature; it's the product.
What We Offer
- Competitive base salary of $175,000–$185,000, plus bonus and equity.
- Unlimited PTO.
- Medical, dental, and vision coverage. BoodleBox covers 75% of premiums.
- 401(k).
- Fully remote work environment.
- Real career growth. Inside a high-growth, mission-driven company.
Salary
$175,000 - $185,000 per year