Show HN: Otto-m8 – A low code AI/ML API deployment Platform
github.comHi all, so I've been working on this low to no code platform that allows you to spin up deep learning workloads(I'm talking LLM's, Huggingface models, etc), interconnect a bunch of them, and deploy them as API's.
The idea essentially came up early in September, when experimenting with combining a Huggingface based BERT model with an LLM at work, and I realized it would be cool if I could do that instantly(especially since it was a prototype). At the time, I was considering a platform that could essentially help you train deep learning models without any code. It was my observation that much of the code required to train or even run inference on HF models have matured significantly. But before I solved that problem, I wanted to solve inference. Initially inspired by n8n and AWS Cloudformation, I built out otto-m8 (translates to automate). Given a json payload that lists out all the resources, and how each model is interconnected, launch it as one-off API the user can query. And thanks to Reactflow, the UI was just something I couldn't just not implement. And as I built it out, I did not want to miss out on the LLM and Agent bit.
With otto-m8, today, you can launch complex workflows by interconnecting HF models and LLM's(currently it supports OpenAI and Ollama only). But I like to see it being more than that. At the core, every workflow is an input process output model. Inputs get processed and there's an output. Therefore, with the way things are setup, one can integrate almost anything and make it interconnectable.
Project Link: https://github.com/farhan0167/otto-m8
Let me know what you guys think. I really would love feedback!
OP here. Thought I'd share a use case that I missed entirely but imagine the following scenario: you've trained a deep learning model and pushed it to Huggingface. While the model is great for what you trained it for, it lives inside a jupyter notebook but you need a deployable API where users can directly interact with your model. Or even better, you want to hook your model up with one of the LLM's out there but you don't want to spend time coding up the API, think about integration code, etc. That's essentially what I'm trying to solve