From AI Curiosity to Cutting-Edge Local LLMs

Read More About Us

How did we get started.

As investors in the AI and Electric Vehicle spaces, we were swept up in the AI hype early this year. I convinced Wenjing to build a custom PC to test some of the new AI models and stay ahead of the curve. After assembling our first computer, it sat idle for a couple of months while we figured out what to do with it.

I experimented with NiceHash Miner, mining around $20 in Bitcoin through CPU mining (Monero(~$0.36/24HR/142W/AMD Ryzen 9 5950X 16-core 32-thread)) and GPU mining (KAWPOW(~$0.60/24HR/440W/EVGA GeForce RTX 3090 FTW3 Ultra Gaming, 24GB GDDR6X) and ETCHash(~$0.39/24HR/440W/EVGA GeForce RTX 3090 FTW3 Ultra Gaming, 24GB GDDR6X)). However, I quickly realized that personal computing mining is not profitable. The returns barely covered electricity costs, and the computer ran extremely hot. After this experience, I decided to abandon mining altogether.

In our next phase, we sought a local large language model that could be run locally. After speaking with several CIOs, it became clear that most businesses take data security very seriously and are hesitant to feed their data into ChatGPT through APIs. We started by exploring HuggingFace's Bloom, but unfortunately, we were unable to get it working.

We then turned to Ollama and Llama 3.0:8B. Ollama proved to be a great choice, with an easy setup process and fast model loading thanks to Open WebUi. You can find our GitHub container here. We began by testing the 8B model with a prompt inspired by Ethan Mollick, generating sentences that ended in "apple." The results were about 50% accurate. However, when we switched to Llama 3.0:70B, accuracy increased to around 90%.

We also tested Stable Diffusion with both models. When using the Llama 3.0:8B model for image generation, the resulting images lacked many elements from the original prompt. In contrast, the Llama 3.0:70B model produced more realistic images that included most of the requested elements.

Of course, there are many other ways to test AI models. We'd love to hear about your favorite methods and invite you to try them out using our portal. If you need a specific model loaded, feel free to ask – we'll do our best to keep up with the latest developments.

When you load a model for the first time, it may take around 1 minute to initialize in memory. However, once the model is loaded, subsequent queries will be answered significantly faster, allowing for a more seamless and responsive experience.

So, what is our current hardware spec?

The Executive Team

Get to know the visionaries steering our ship. Our executive team blends innovation with expertise, driving us forward with passion and precision.

Junfeng Ma, PMP

Founder

As the founder who kicked off our journey with a passion for AI and tech, I’m thrilled to share the story behind our innovative start and the vision that drives us forward.

Wenjing Du, PMP

Founder

Leading our tech vision, I am dedicated to pioneering AI solutions that drive innovation and redefine what's possible in the tech landscape.