Running OramaCore

OramaCore is currently under active development. Our goal is to release the first Beta version (v0.1.0) on Jan 31st, 2025, and the first stable version (v1.0.0) on Feb 28th, 2025.

While the system is already quite stable, please note that APIs will undergo changes in v0.1.0 and v1.0.0.

Using Docker

The simplest way to get started is by pulling the official Docker image from DockerHub:

docker pull oramasearch/oramacore:latest

Create a config.yaml configuration file and then run the Docker image:

docker run \
  -p 8080:8080 \
  -v ${HOME}/.cache/huggingface:/root/.cache/huggingface \
  -v ./config.yaml:/app/config.yaml \
  --gpus all \
  oramacore

As shown above, we're using the --gpus all flag to instruct Docker to utilize all the available GPUs on your machine.

To use this flag, ensure that the NVIDIA Container Toolkit is installed on your system:

distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
 
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
 
# Remember to restart Docker if it's already installed.
sudo systemctl restart docker

Building from source

You can also build OramaCore from source by cloning the repository from GitHub:

git clone https://github.com/oramasearch/orama-core

The project consists of two parts: a Rust core and a Python server.

The Python server is responsible for generating embeddings, interacting with local LLMs, and more. It communicates with the Rust core using gRPC.

To build the entire system, ensure that you have Rust installed (installation guide) and Python (recommended version: 3.11).

Building Rust

Simply run the following command from the root directory:

cargo build --release

This will generate a binary located in /target/release/oramacore.

Building Python

Navigate to the src/ai_server directory and install the required dependencies. You'll find two distinct requirements files:

requirements.txt
requirements-cpu.txt

The first file contains dependencies for GPU usage, which we highly recommend for production with an NVIDIA GPU.

If you are running OramaCore on a system without an NVIDIA GPU (e.g., a Mac), use requirements-cpu.txt.

Before installing, create a virtual environment:

python3.11 -m venv .venv
source .venv/bin/activate

Then, install the dependencies:

pip install -r requirements.txt # or pip install -r requirements-cpu.txt

When you run the server, OramaCore will automatically download the required models specified in the configuration file.

The download time will depend on your internet connection.

Starting the OramaCore server

After installing the dependencies and compiling the binaries, you'll need to start two separate services.

In one terminal tab, run the Python server inside of src/ai_server:

python server.py

Once the process started, run the Rust core binary:

./target/release/oramacore

In future versions of OramaCore, we plan to unify everything into a single binary, so you won't need to run two separate processes manually.