Configurable embedding dimensions for Marengo 3.5
Choose how many dimensions Marengo 3.5 uses for each embedding by setting the embedding_dimension parameter to 128, 256, or 512. The default is 512, so existing integrations are unaffected. Use the same value across an index.
A shorter embedding consists of the first values of the full-length embedding. Shorter embeddings reduce the size of your index and speed up similarity search; longer embeddings improve retrieval quality.
To use this feature, upgrade to the latest version of the Python or Node.js SDK:
For details, see the Create sync embeddings and Create an async embedding task pages in the API Reference section, or the SDK reference for Python and Node.js.
New segmentation options for asynchronous document embeddings
Control how the platform divides a document before it creates embeddings by setting the segmentation parameter. The options depend on the file type:
- PDF files: Divide each page into a 2×2 grid, producing five embeddings per page: one for the whole page, and one for each quarter.
- Plain text and Markdown files: Divide the file into chunks of whole sentences, with a chunk size you control.
To use this feature, upgrade to the latest version of the Python or Node.js SDK:
For details, see the Create an async embedding task page in the API Reference section, or the SDK reference for Python and Node.js.