# 4MINDS OpenAI-Compatible API Reference
Source: https://docs.4minds.ai/4-minds-open-ai-compatible-api-reference
Complete reference for 4MINDS OpenAI-compatible REST API endpoints.
> **Note:** This API is OpenAI-compatible. You can use the official OpenAI SDK (Python or JavaScript) by pointing `base_url` / `baseURL` to `https://api.4minds.ai/v1` and providing your 4MINDS API key.
***
## Base URL
```text theme={null}
https://api.4minds.ai
```
***
## Models
OpenAI-compatible models API. List, retrieve, and delete your fine-tuned models.
### List All Available Models
**GET** `/v1/models`
List all available models (OpenAI-compatible).
```bash cURL theme={null}
curl -X GET https://api.4minds.ai/v1/models \
-H "Authorization: Bearer YOUR_API_KEY"
```
```python Python (openai) theme={null}
from openai import OpenAI
client = OpenAI(
api_key="YOUR_API_KEY",
base_url="https://api.4minds.ai/v1"
)
# List all models
models = client.models.list()
for model in models.data:
print(f"Model: {model.id}")
print(f" Owner: {model.owned_by}")
print(f" Created: {model.created}")
```
```python Python (requests) theme={null}
import requests
url = "https://api.4minds.ai/v1/models"
headers = {
"Authorization": "Bearer YOUR_API_KEY"
}
response = requests.get(url, headers=headers)
result = response.json()
for model in result['data']:
print(f"Model: {model['id']}")
print(f" Owner: {model['owned_by']}")
```
```javascript JavaScript theme={null}
import OpenAI from 'openai';
const openai = new OpenAI({
apiKey: 'YOUR_API_KEY',
baseURL: 'https://api.4minds.ai/v1'
});
async function listModels() {
const models = await openai.models.list();
for (const model of models.data) {
console.log(`Model: ${model.id}`);
console.log(` Owner: ${model.owned_by}`);
}
}
listModels();
```
```javascript Node.js theme={null}
const OpenAI = require('openai');
const openai = new OpenAI({
apiKey: 'YOUR_API_KEY',
baseURL: 'https://api.4minds.ai/v1'
});
async function listModels() {
const models = await openai.models.list();
for (const model of models.data) {
console.log(`Model: ${model.id}`);
console.log(` Owner: ${model.owned_by}`);
}
}
listModels();
```
***
### Get Model Details
**GET** `/v1/models/{model_id}`
Get details about a specific model (OpenAI-compatible).
#### Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | ---------------------------------- |
| model\_id | string | Yes | The unique identifier of the model |
```bash cURL theme={null}
curl -X GET https://api.4minds.ai/v1/models/{model_id} \
-H "Authorization: Bearer YOUR_API_KEY"
```
```python Python (openai) theme={null}
from openai import OpenAI
client = OpenAI(
api_key="YOUR_API_KEY",
base_url="https://api.4minds.ai/v1"
)
model = client.models.retrieve("your-model-id")
print(f"Model: {model.id}")
print(f"Owner: {model.owned_by}")
```
```javascript JavaScript theme={null}
import OpenAI from 'openai';
const openai = new OpenAI({
apiKey: 'YOUR_API_KEY',
baseURL: 'https://api.4minds.ai/v1'
});
const model = await openai.models.retrieve('your-model-id');
console.log(`Model: ${model.id}`);
console.log(`Owner: ${model.owned_by}`);
```
***
### Delete a Model
**DELETE** `/v1/models/{model_id}`
Delete a fine-tuned model (OpenAI-compatible).
#### Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | -------------------------------------------- |
| model\_id | string | Yes | The unique identifier of the model to delete |
```bash cURL theme={null}
curl -X DELETE https://api.4minds.ai/v1/models/{model_id} \
-H "Authorization: Bearer YOUR_API_KEY"
```
```python Python (openai) theme={null}
from openai import OpenAI
client = OpenAI(
api_key="YOUR_API_KEY",
base_url="https://api.4minds.ai/v1"
)
client.models.delete("your-model-id")
print("Model deleted successfully")
```
```javascript JavaScript theme={null}
import OpenAI from 'openai';
const openai = new OpenAI({
apiKey: 'YOUR_API_KEY',
baseURL: 'https://api.4minds.ai/v1'
});
await openai.models.delete('your-model-id');
console.log('Model deleted successfully');
```
***
## Chat Completions
OpenAI-compatible chat completions with 4minds Constellations extensions. Supports streaming, agent status events, multi-hop planning, and RAG context retrieval. Stored completions can be listed, retrieved, updated, and deleted.
### Create a Chat Completion
**POST** `/v1/chat/completions`
Create a chat completion (OpenAI-compatible with 4minds extensions).
```bash cURL (Basic) theme={null}
curl -X POST https://api.4minds.ai/v1/chat/completions \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "4minds-model-123",
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is machine learning?"}
],
"temperature": 0.7,
"max_tokens": 500
}'
```
```bash cURL (Streaming) theme={null}
curl -X POST https://api.4minds.ai/v1/chat/completions \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "4minds-model-123",
"messages": [
{"role": "user", "content": "Explain neural networks"}
],
"stream": true
}'
```
```python Python theme={null}
from openai import OpenAI
client = OpenAI(
api_key="YOUR_API_KEY",
base_url="https://api.4minds.ai/v1"
)
# Basic chat completion
response = client.chat.completions.create(
model="4minds-model-123",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is machine learning?"}
],
temperature=0.7,
max_tokens=500
)
print(response.choices[0].message.content)
```
```python Python (Streaming) theme={null}
from openai import OpenAI
client = OpenAI(
api_key="YOUR_API_KEY",
base_url="https://api.4minds.ai/v1"
)
# Streaming chat completion
stream = client.chat.completions.create(
model="4minds-model-123",
messages=[
{"role": "user", "content": "Explain neural networks"}
],
stream=True
)
for chunk in stream:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)
```
```javascript JavaScript theme={null}
import OpenAI from 'openai';
const openai = new OpenAI({
apiKey: 'YOUR_API_KEY',
baseURL: 'https://api.4minds.ai/v1'
});
async function chat() {
const response = await openai.chat.completions.create({
model: '4minds-model-123',
messages: [
{ role: 'system', content: 'You are a helpful assistant.' },
{ role: 'user', content: 'What is machine learning?' }
],
temperature: 0.7,
max_tokens: 500
});
console.log(response.choices[0].message.content);
}
chat();
```
```javascript Node.js theme={null}
const OpenAI = require('openai');
const openai = new OpenAI({
apiKey: 'YOUR_API_KEY',
baseURL: 'https://api.4minds.ai/v1'
});
async function chat() {
const response = await openai.chat.completions.create({
model: '4minds-model-123',
messages: [
{ role: 'system', content: 'You are a helpful assistant.' },
{ role: 'user', content: 'What is machine learning?' }
],
temperature: 0.7,
max_tokens: 500
});
console.log(response.choices[0].message.content);
}
chat();
```
***
### List Stored Chat Completions
**GET** `/v1/chat/completions`
List stored chat completions with optional filtering by model.
```bash cURL theme={null}
curl -X GET "https://api.4minds.ai/v1/chat/completions?model=4minds-model-123" \
-H "Authorization: Bearer YOUR_API_KEY"
```
***
### Get a Stored Chat Completion
**GET** `/v1/chat/completions/{completion_id}`
Retrieve a stored chat completion by ID.
#### Path Parameters
| Parameter | Type | Required | Description |
| -------------- | ------ | -------- | ---------------------------------------------- |
| completion\_id | string | Yes | The unique identifier of the stored completion |
```bash cURL theme={null}
curl -X GET https://api.4minds.ai/v1/chat/completions/{completion_id} \
-H "Authorization: Bearer YOUR_API_KEY"
```
***
### Update a Stored Chat Completion
**POST** `/v1/chat/completions/{completion_id}`
Update a stored chat completion (e.g. rename or set metadata).
#### Path Parameters
| Parameter | Type | Required | Description |
| -------------- | ------ | -------- | ---------------------------------------------- |
| completion\_id | string | Yes | The unique identifier of the stored completion |
```bash cURL theme={null}
curl -X POST https://api.4minds.ai/v1/chat/completions/{completion_id} \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"title": "Updated Title"}'
```
***
### Delete a Stored Chat Completion
**DELETE** `/v1/chat/completions/{completion_id}`
Delete a stored chat completion.
#### Path Parameters
| Parameter | Type | Required | Description |
| -------------- | ------ | -------- | ---------------------------------------------- |
| completion\_id | string | Yes | The unique identifier of the stored completion |
```bash cURL theme={null}
curl -X DELETE https://api.4minds.ai/v1/chat/completions/{completion_id} \
-H "Authorization: Bearer YOUR_API_KEY"
```
***
### List Messages from a Stored Chat Completion
**GET** `/v1/chat/completions/{completion_id}/messages`
List messages from a stored chat completion.
#### Path Parameters
| Parameter | Type | Required | Description |
| -------------- | ------ | -------- | ---------------------------------------------- |
| completion\_id | string | Yes | The unique identifier of the stored completion |
```bash cURL theme={null}
curl -X GET https://api.4minds.ai/v1/chat/completions/{completion_id}/messages \
-H "Authorization: Bearer YOUR_API_KEY"
```
***
## Files
OpenAI-compatible file management API. Upload, list, retrieve, and delete files for fine-tuning. 4minds extends OpenAI's single-file upload with multi-file datasets — upload multiple files to a single dataset for training.
### Upload a File
**POST** `/v1/files`
Upload a file for fine-tuning (OpenAI-compatible with multi-file extension).
The response includes a `dataset_id` — use it to upload more files to the same dataset.
**Optional `4minds` form field (JSON string):**
| Field | Type | Description |
| -------------- | ------- | ------------------------------------------------------- |
| dataset\_name | string | Custom name for the new dataset |
| dataset\_id | integer | Upload to an existing dataset |
| training\_type | string | `"graph"` (default), `"rl"`, or `"sft"` |
| model\_params | object | Custom hyperparameters, e.g. `{"learning_rate": 0.001}` |
**Behavior:**
| Scenario | Result |
| ----------------------------- | ------------------------------------ |
| Omit `4minds` entirely | New dataset with auto-generated name |
| `dataset_name` only | New dataset with your custom name |
| `dataset_id` only | Upload to existing dataset |
| `dataset_id` + `dataset_name` | `dataset_id` takes priority |
> **Note:** If the dataset already has a model, training is triggered automatically on upload.
```bash cURL theme={null}
# Standard OpenAI file upload
curl -X POST https://api.4minds.ai/v1/files \
-H "Authorization: Bearer YOUR_API_KEY" \
-F "file=@training_data.jsonl" \
-F "purpose=fine-tune"
# With optional 4minds extensions (omit for standard OpenAI behavior)
curl -X POST https://api.4minds.ai/v1/files \
-H "Authorization: Bearer YOUR_API_KEY" \
-F "file=@training_data.jsonl" \
-F "purpose=fine-tune" \
-F '4minds={"dataset_name": "My Training Data", "training_type": "graph"}'
```
```bash cURL (Multi-File Upload) theme={null}
# Upload multiple files to the SAME dataset
# Step 1: Upload first file — creates a new dataset
curl -X POST https://api.4minds.ai/v1/files \
-H "Authorization: Bearer YOUR_API_KEY" \
-F "file=@document1.pdf" \
-F "purpose=fine-tune" \
-F '4minds={"dataset_name": "My Training Data"}'
# Response: {"id": "file-abc123...", "dataset_id": 1234, ...}
# Step 2: Upload more files to the SAME dataset using dataset_id
curl -X POST https://api.4minds.ai/v1/files \
-H "Authorization: Bearer YOUR_API_KEY" \
-F "file=@document2.pdf" \
-F "purpose=fine-tune" \
-F '4minds={"dataset_id": 1234}'
# Step 3: Upload with custom training type and hyperparameters
curl -X POST https://api.4minds.ai/v1/files \
-H "Authorization: Bearer YOUR_API_KEY" \
-F "file=@document3.pdf" \
-F "purpose=fine-tune" \
-F '4minds={"dataset_id": 1234, "training_type": "sft", "model_params": {"learning_rate": 0.001}}'
# Step 4: Create a fine-tuning job using any file from the dataset
# Training processes ALL files in the dataset
curl -X POST https://api.4minds.ai/v1/fine_tuning/jobs \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "Gemma-12B AWQ",
"training_file": "file-abc123...",
"suffix": "my-custom-model"
}'
```
```python Python theme={null}
from openai import OpenAI
client = OpenAI(
api_key="YOUR_API_KEY",
base_url="https://api.4minds.ai/v1"
)
# Standard OpenAI file upload
with open("training_data.jsonl", "rb") as f:
file = client.files.create(
file=f,
purpose="fine-tune"
)
print(f"File ID: {file.id}")
print(f"Filename: {file.filename}")
print(f"Size: {file.bytes} bytes")
# The response includes dataset_id — use it to upload
# more files to the same dataset via the requests library.
#
# Optional '4minds' parameter fields:
# dataset_name (str) - Custom name for the new dataset
# dataset_id (int) - Upload to an existing dataset
# training_type (str) - "graph" (default), "rl", or "sft"
# model_params (dict) - e.g. {"learning_rate": 0.001}
```
```python Python (Multi-File Upload) theme={null}
import requests
import json
api_key = "YOUR_API_KEY"
base_url = "https://api.4minds.ai/v1/files"
headers = {"Authorization": f"Bearer {api_key}"}
# Step 1: Upload first file with a custom dataset name
with open("document1.pdf", "rb") as f:
response = requests.post(
base_url,
headers=headers,
files={"file": f},
data={
"purpose": "fine-tune",
"4minds": json.dumps({
"dataset_name": "Medical Research Data",
"training_type": "graph"
})
}
)
result = response.json()
dataset_id = result["dataset_id"]
print(f"Dataset created: {dataset_id}")
print(f"File ID: {result['id']}")
print(f"Status: {result['status']}")
# Step 2: Upload more files to the SAME dataset
for filename in ["document2.pdf", "document3.pdf"]:
with open(filename, "rb") as f:
response = requests.post(
base_url,
headers=headers,
files={"file": f},
data={
"purpose": "fine-tune",
"4minds": json.dumps({"dataset_id": dataset_id})
}
)
print(f"Uploaded {filename}: {response.json()['id']}")
# Step 3: Create a fine-tuning job — trains ALL files in dataset
from openai import OpenAI
client = OpenAI(api_key=api_key, base_url="https://api.4minds.ai/v1")
job = client.fine_tuning.jobs.create(
model="Gemma-12B AWQ",
training_file=result["id"],
suffix="my-custom-model"
)
print(f"Fine-tuning job: {job.id}, Status: {job.status}")
```
```javascript JavaScript theme={null}
import OpenAI from 'openai';
import fs from 'fs';
const openai = new OpenAI({
apiKey: 'YOUR_API_KEY',
baseURL: 'https://api.4minds.ai/v1'
});
// Standard OpenAI file upload
const file = await openai.files.create({
file: fs.createReadStream('training_data.jsonl'),
purpose: 'fine-tune'
});
console.log(`File ID: ${file.id}`);
console.log(`Filename: ${file.filename}`);
console.log(`Size: ${file.bytes} bytes`);
// The response includes dataset_id — use it to upload
// more files to the same dataset via fetch().
//
// Optional '4minds' form field (JSON string):
// dataset_name (string) - Custom name for the new dataset
// dataset_id (integer) - Upload to an existing dataset
// training_type (string) - "graph" (default), "rl", or "sft"
// model_params (object) - e.g. {learning_rate: 0.001}
```
```javascript JavaScript (Multi-File Upload) theme={null}
// Upload multiple files to one dataset
const apiKey = 'YOUR_API_KEY';
const baseUrl = 'https://api.4minds.ai/v1/files';
// Step 1: Upload first file with a custom dataset name
const formData1 = new FormData();
formData1.append('file', fileInput.files[0]);
formData1.append('purpose', 'fine-tune');
formData1.append('4minds', JSON.stringify({
dataset_name: 'My Training Data',
training_type: 'graph'
}));
const response1 = await fetch(baseUrl, {
method: 'POST',
headers: { 'Authorization': `Bearer ${apiKey}` },
body: formData1
});
const result1 = await response1.json();
const datasetId = result1.dataset_id;
console.log(`Dataset created: ${datasetId}`);
console.log(`Status: ${result1.status}`);
// Step 2: Upload more files to the SAME dataset
for (const file of additionalFiles) {
const formData = new FormData();
formData.append('file', file);
formData.append('purpose', 'fine-tune');
formData.append('4minds', JSON.stringify({
dataset_id: datasetId
}));
const response = await fetch(baseUrl, {
method: 'POST',
headers: { 'Authorization': `Bearer ${apiKey}` },
body: formData
});
const result = await response.json();
console.log(`Uploaded ${result.id} (status: ${result.status})`);
}
```
```javascript Node.js theme={null}
const OpenAI = require('openai');
const fs = require('fs');
const openai = new OpenAI({
apiKey: 'YOUR_API_KEY',
baseURL: 'https://api.4minds.ai/v1'
});
async function uploadFile() {
// Standard OpenAI file upload
const file = await openai.files.create({
file: fs.createReadStream('training_data.jsonl'),
purpose: 'fine-tune'
});
console.log(`File ID: ${file.id}`);
console.log(`Filename: ${file.filename}`);
console.log(`Size: ${file.bytes} bytes`);
// The response includes dataset_id — use it to upload
// more files to the same dataset.
//
// Optional '4minds' form field (JSON string):
// dataset_name (string) - Custom name for the new dataset
// dataset_id (integer) - Upload to an existing dataset
// training_type (string) - "graph" (default), "rl", or "sft"
// model_params (object) - e.g. {learning_rate: 0.001}
}
uploadFile();
```
```javascript Node.js (Multi-File Upload) theme={null}
const fs = require('fs');
const path = require('path');
const apiKey = 'YOUR_API_KEY';
const baseUrl = 'https://api.4minds.ai/v1/files';
async function uploadMultipleFiles() {
// Step 1: Upload first file with a custom dataset name
const form1 = new FormData();
form1.append('file', new Blob([fs.readFileSync('document1.pdf')]), 'document1.pdf');
form1.append('purpose', 'fine-tune');
form1.append('4minds', JSON.stringify({
dataset_name: 'My Training Data',
training_type: 'graph'
}));
const response1 = await fetch(baseUrl, {
method: 'POST',
headers: { 'Authorization': `Bearer ${apiKey}` },
body: form1
});
const result1 = await response1.json();
const datasetId = result1.dataset_id;
console.log(`Dataset created: ${datasetId}`);
console.log(`First file: ${result1.id}`);
// Step 2: Upload more files to the SAME dataset
const moreFiles = ['document2.pdf', 'document3.pdf'];
for (const filename of moreFiles) {
const form = new FormData();
form.append('file', new Blob([fs.readFileSync(filename)]), filename);
form.append('purpose', 'fine-tune');
form.append('4minds', JSON.stringify({ dataset_id: datasetId }));
const response = await fetch(baseUrl, {
method: 'POST',
headers: { 'Authorization': `Bearer ${apiKey}` },
body: form
});
const result = await response.json();
console.log(`Uploaded ${filename}: ${result.id} (status: ${result.status})`);
}
// Step 3: Create a fine-tuning job — trains ALL files in dataset
const OpenAI = require('openai');
const openai = new OpenAI({ apiKey, baseURL: 'https://api.4minds.ai/v1' });
const job = await openai.fineTuning.jobs.create({
model: 'Gemma-12B AWQ',
training_file: result1.id,
suffix: 'my-custom-model'
});
console.log(`Fine-tuning job: ${job.id}, Status: ${job.status}`);
}
uploadMultipleFiles();
```
***
### List All Uploaded Files
**GET** `/v1/files`
List all uploaded files with optional purpose filter (OpenAI-compatible).
```bash cURL theme={null}
curl -X GET "https://api.4minds.ai/v1/files?purpose=fine-tune" \
-H "Authorization: Bearer YOUR_API_KEY"
```
***
### Get File Details
**GET** `/v1/files/{file_id}`
Retrieve details about a specific file (OpenAI-compatible).
#### Path Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | --------------------------------- |
| file\_id | string | Yes | The unique identifier of the file |
```bash cURL theme={null}
curl -X GET https://api.4minds.ai/v1/files/{file_id} \
-H "Authorization: Bearer YOUR_API_KEY"
```
***
### Delete a File
**DELETE** `/v1/files/{file_id}`
Delete a file (OpenAI-compatible).
#### Path Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | ------------------------------------------- |
| file\_id | string | Yes | The unique identifier of the file to delete |
```bash cURL theme={null}
curl -X DELETE https://api.4minds.ai/v1/files/{file_id} \
-H "Authorization: Bearer YOUR_API_KEY"
```
***
## Fine-Tuning
OpenAI-compatible fine-tuning API. Create, monitor, and manage fine-tuning jobs to customize models with your training data.
> **Note:** Training processes ALL files in the dataset, not just the file referenced by `training_file`. Use `external_model_id` to deploy to an external model registered via `GET /v1/external-models`.
### Create a Fine-Tuning Job
**POST** `/v1/fine_tuning/jobs`
Create a fine-tuning job to train a model (OpenAI-compatible). Optionally pass `external_model_id` to target an external model from a connected integration.
#### Request Body
| Parameter | Type | Required | Description |
| ------------------- | ------- | -------- | --------------------------------------------------------- |
| model | string | Yes | The base model name to fine-tune (e.g. `"Gemma-12B AWQ"`) |
| training\_file | string | Yes | The file ID to use for training |
| suffix | string | No | Custom suffix for the fine-tuned model name |
| external\_model\_id | integer | No | ID of an external model from a connected integration |
```bash cURL theme={null}
# Create fine-tuning job — trains ALL files in the dataset
curl -X POST https://api.4minds.ai/v1/fine_tuning/jobs \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "Gemma-12B AWQ",
"training_file": "file-abc123def456",
"suffix": "my-custom-model"
}'
# With an external model (from connected integrations)
curl -X POST https://api.4minds.ai/v1/fine_tuning/jobs \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "Gemma-12B AWQ",
"training_file": "file-abc123def456",
"suffix": "my-custom-model",
"external_model_id": 42
}'
```
```python Python theme={null}
from openai import OpenAI
client = OpenAI(
api_key="YOUR_API_KEY",
base_url="https://api.4minds.ai/v1"
)
# Create fine-tuning job
job = client.fine_tuning.jobs.create(
model="Gemma-12B AWQ",
training_file="file-abc123def456",
suffix="my-custom-model"
)
print(f"Job ID: {job.id}")
print(f"Status: {job.status}")
print(f"Model: {job.fine_tuned_model}")
# With an external model (from connected integrations):
# job = client.fine_tuning.jobs.create(
# model="Gemma-12B AWQ",
# training_file="file-abc123def456",
# suffix="my-custom-model",
# extra_body={"external_model_id": 42}
# )
```
```javascript JavaScript theme={null}
import OpenAI from 'openai';
const openai = new OpenAI({
apiKey: 'YOUR_API_KEY',
baseURL: 'https://api.4minds.ai/v1'
});
async function createFineTuningJob() {
const job = await openai.fineTuning.jobs.create({
model: 'Gemma-12B AWQ',
training_file: 'file-abc123def456',
suffix: 'my-custom-model'
});
console.log(`Job ID: ${job.id}`);
console.log(`Status: ${job.status}`);
console.log(`Model: ${job.fine_tuned_model}`);
// With an external model (from connected integrations):
// Pass external_model_id in the body to target an external model
// const job2 = await openai.fineTuning.jobs.create({
// model: 'Gemma-12B AWQ',
// training_file: 'file-abc123def456',
// suffix: 'my-custom-model',
// body: { external_model_id: 42 }
// });
}
createFineTuningJob();
```
```javascript Node.js theme={null}
const OpenAI = require('openai');
const openai = new OpenAI({
apiKey: 'YOUR_API_KEY',
baseURL: 'https://api.4minds.ai/v1'
});
async function createFineTuningJob() {
const job = await openai.fineTuning.jobs.create({
model: 'Gemma-12B AWQ',
training_file: 'file-abc123def456',
suffix: 'my-custom-model'
});
console.log(`Job ID: ${job.id}`);
console.log(`Status: ${job.status}`);
console.log(`Model: ${job.fine_tuned_model}`);
// With an external model (from connected integrations):
// Pass external_model_id in the body to target an external model
// const job2 = await openai.fineTuning.jobs.create({
// model: 'Gemma-12B AWQ',
// training_file: 'file-abc123def456',
// suffix: 'my-custom-model',
// body: { external_model_id: 42 }
// });
}
createFineTuningJob();
```
***
### List All Fine-Tuning Jobs
**GET** `/v1/fine_tuning/jobs`
List all fine-tuning jobs (OpenAI-compatible).
```bash cURL theme={null}
curl -X GET https://api.4minds.ai/v1/fine_tuning/jobs \
-H "Authorization: Bearer YOUR_API_KEY"
```
***
### Get Fine-Tuning Job Details
**GET** `/v1/fine_tuning/jobs/{job_id}`
Get fine-tuning job details (OpenAI-compatible).
#### Path Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | -------------------------------------------- |
| job\_id | string | Yes | The unique identifier of the fine-tuning job |
```bash cURL theme={null}
curl -X GET https://api.4minds.ai/v1/fine_tuning/jobs/{job_id} \
-H "Authorization: Bearer YOUR_API_KEY"
```
***
### List Fine-Tune Categories
**GET** `/v1/fine-tune-categories`
List available fine-tune categories for domain-specific prompting (4minds extension).
```bash cURL theme={null}
curl -X GET https://api.4minds.ai/v1/fine-tune-categories \
-H "Authorization: Bearer YOUR_API_KEY"
```
***
### Cancel a Fine-Tuning Job
**POST** `/v1/fine_tuning/jobs/{job_id}/cancel`
Cancel a running fine-tuning job (OpenAI-compatible).
#### Path Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | ------------------------------------------------------ |
| job\_id | string | Yes | The unique identifier of the fine-tuning job to cancel |
```bash cURL theme={null}
curl -X POST https://api.4minds.ai/v1/fine_tuning/jobs/{job_id}/cancel \
-H "Authorization: Bearer YOUR_API_KEY"
```
***
## Datasets (OpenAI-Compatible Extension)
OpenAI-compatible datasets API. Create, list, retrieve, delete, and import datasets. Datasets group training files together for fine-tuning jobs. You can also import datasets directly from connected integrations.
### List All Datasets
**GET** `/v1/datasets`
List all datasets with their file counts and status.
```bash cURL theme={null}
curl -X GET https://api.4minds.ai/v1/datasets \
-H "Authorization: Bearer YOUR_API_KEY"
# Example response:
# {
# "object": "list",
# "data": [
# {
# "id": 1234,
# "name": "Medical Research Data",
# "file_count": 3,
# "total_bytes": 524288,
# "status": "ready",
# "created_at": "2025-03-01T12:00:00Z"
# }
# ]
# }
```
```python Python theme={null}
import requests
url = "https://api.4minds.ai/v1/datasets"
headers = {"Authorization": "Bearer YOUR_API_KEY"}
response = requests.get(url, headers=headers)
result = response.json()
for dataset in result["data"]:
print(f"ID: {dataset['id']}")
print(f" Name: {dataset['name']}")
print(f" Files: {dataset['file_count']}")
print(f" Status: {dataset['status']}")
```
```javascript JavaScript theme={null}
const response = await fetch('https://api.4minds.ai/v1/datasets', {
headers: { 'Authorization': 'Bearer YOUR_API_KEY' }
});
const result = await response.json();
for (const dataset of result.data) {
console.log(`ID: ${dataset.id}`);
console.log(` Name: ${dataset.name}`);
console.log(` Files: ${dataset.file_count}`);
console.log(` Status: ${dataset.status}`);
}
```
```javascript Node.js theme={null}
const response = await fetch('https://api.4minds.ai/v1/datasets', {
headers: { 'Authorization': 'Bearer YOUR_API_KEY' }
});
const result = await response.json();
for (const dataset of result.data) {
console.log(`ID: ${dataset.id}`);
console.log(` Name: ${dataset.name}`);
console.log(` Files: ${dataset.file_count}`);
console.log(` Status: ${dataset.status}`);
}
```
***
### Create a New Dataset
**POST** `/v1/datasets`
Create a new empty dataset to group training files.
```bash cURL theme={null}
curl -X POST https://api.4minds.ai/v1/datasets \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"name": "My New Dataset"}'
```
***
### Import a Dataset
**POST** `/v1/datasets/import`
Import a dataset directly from a connected integration (e.g. Databricks, S3, Azure Blob).
```bash cURL theme={null}
curl -X POST https://api.4minds.ai/v1/datasets/import \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"integration_id": "your-integration-id",
"source_path": "s3://your-bucket/your-data/"
}'
```
***
### Get Dataset Details
**GET** `/v1/datasets/{id}`
Get details of a specific dataset including its files.
#### Path Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | ------------------------------------ |
| id | string | Yes | The unique identifier of the dataset |
```bash cURL theme={null}
curl -X GET https://api.4minds.ai/v1/datasets/{id} \
-H "Authorization: Bearer YOUR_API_KEY"
```
***
### Delete a Dataset
**DELETE** `/v1/datasets/{id}`
Delete a dataset and optionally its associated files.
#### Path Parameters
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | ------------------------------------ |
| id | string | Yes | The unique identifier of the dataset |
```bash cURL theme={null}
curl -X DELETE https://api.4minds.ai/v1/datasets/{id} \
-H "Authorization: Bearer YOUR_API_KEY"
```
***
## Authentication
All API requests require authentication using a Bearer token in the Authorization header:
```text theme={null}
Authorization: Bearer YOUR_API_KEY
```
You can obtain your API key from your 4MINDS dashboard.
***
## OpenAI SDK Quick Setup
Since this API is OpenAI-compatible, you can use the official OpenAI SDK by setting the `base_url` to `https://api.4minds.ai/v1`:
```python Python theme={null}
from openai import OpenAI
client = OpenAI(
api_key="YOUR_API_KEY",
base_url="https://api.4minds.ai/v1"
)
```
```javascript JavaScript / Node.js theme={null}
import OpenAI from 'openai'; // or: const OpenAI = require('openai');
const openai = new OpenAI({
apiKey: 'YOUR_API_KEY',
baseURL: 'https://api.4minds.ai/v1'
});
```
***
## Response Format
All responses are returned in JSON format and follow OpenAI-compatible response structures.
### Success Response
```json theme={null}
{
"object": "list",
"data": [ ... ]
}
```
### Error Response
```json theme={null}
{
"error": {
"message": "Error description",
"type": "invalid_request_error",
"code": "error_code"
}
}
```
***
# 4MINDS API Overview
Source: https://docs.4minds.ai/api-overview
Access 4MINDS programmatically through REST API endpoints.
**The Synthesis Graph™ is not directly accessible via API.** There is no graph query endpoint — the knowledge graph is queried internally during model inference. To retrieve knowledge from the graph, call a model inference endpoint; the model traverses the graph as part of generating its response.
**Using "Try it" on the endpoint pages.** Each endpoint under the **4MINDS API Endpoints** group in the left sidebar includes a live "Try it" panel. Before sending a request, expand the **Authorization** section on the right and paste your API key into the **Bearer Token** field — otherwise requests return `401 Unauthorized`. Your key is stored locally in your browser and sent only to `https://api.4minds.ai`. Get your key from the **API** tab in the 4MINDS platform.
## **Accessing 4MINDS API**
The **API** is a dedicated tab in the main navigation. It's available on the **Enterprise plan** and only visible to users with **Admin** rights — if you don't see the tab, confirm your plan and role with your organization administrator.
From the **API** tab, you can view your API keys, configure access, and browse the API reference.
![Screen Shot2025 11 06at6 51 31PM Pn]( "Screen Shot2025 11 06at6 51 31PM Pn")
![Screen Shot2025 11 06at6 51 31PM Pn]( "Screen Shot2025 11 06at6 51 31PM Pn")
The '**Quick Start**' section in the left sidebar lets you select the API version and a resource from the dropdowns to view all available 4MINDS API endpoints.
4MINDS offers two API versions, selectable from the first dropdown in the **Quick Start** section: the **regular 4MINDS API** and the **OpenAI-compatible API**. Use the OpenAI-compatible version if you're migrating from OpenAI or want to reuse existing OpenAI SDK code with minimal changes.
The API enables programmatic interaction with your ***models***, ***conversations***, ***datasets***, ***evaluations***, ***inference, use cases, fine-tuning categories, personas, external models,*** *and **integrations***.
Located next to the '**Quick Start**' section, the '**Code Examples**' section provides ready-to-use code snippets in cURL, Python, JavaScript, and Node.js. Simply select your preferred language from the dropdown for any endpoint.
![Screen Shot2025 11 06at7 02 58PM Pn]()
![Screen Shot2025 11 06at7 02 58PM Pn]()
The table below the '**Quick Start**' section contains resource IDs required for certain API requests. Available resources in the dropdown include: ***models***, ***base models***, ***datasets***, ***training types***, ***use cases***, ***fine-tuning categories*** and ***personas***.
![Screen Shot2025 11 06at7 11 15PM Pn]()
![Screen Shot2025 11 06at7 11 15PM Pn]()
## Authentication
To start working with the 4MINDS API, click the '**Create API Key**' button to generate your authentication key.
![Screen Shot2025 11 06at7 20 24PM Pn]( "Screen Shot2025 11 06at7 20 24PM Pn")
![Screen Shot2025 11 06at7 20 24PM Pn]( "Screen Shot2025 11 06at7 20 24PM Pn")
Configure your API key by entering a name, selecting the key type (Universal or Model-specific), setting permissions (All Access, Read Only, Inference Only, or Custom) and expiration period. Click the '**Create Key**' button to finalize and generate your authentication key.
![Screen Shot2025 11 06at7 28 00PM Pn]( "Screen Shot2025 11 06at7 28 00PM Pn")
![Screen Shot2025 11 06at7 28 00PM Pn]( "Screen Shot2025 11 06at7 28 00PM Pn")
You'll receive a confirmation that your key has been created successfully.
Copy your API authentication key now and store it securely, as you won't be able to retrieve it later.
![Screen Shot2025 11 06at7 31 06PM Pn]( "Screen Shot2025 11 06at7 31 06PM Pn")
![Screen Shot2025 11 06at7 31 06PM Pn]( "Screen Shot2025 11 06at7 31 06PM Pn")
## Base URL
```text theme={null}
https://api.4minds.ai
```
## Support
For API access or questions, contact support at [support@4minds.ai](mailto:support@4minds.ai).
# Create a conversation
Source: https://docs.4minds.ai/api-reference/conversations/create-a-conversation
/api-reference/openapi.json post /api/v1/user/conversations
# Delete a conversation
Source: https://docs.4minds.ai/api-reference/conversations/delete-a-conversation
/api-reference/openapi.json delete /api/v1/user/conversations/{conversation_id}
# List conversations for a model
Source: https://docs.4minds.ai/api-reference/conversations/list-conversations-for-a-model
/api-reference/openapi.json get /api/v1/user/conversations
# List messages in a conversation
Source: https://docs.4minds.ai/api-reference/conversations/list-messages-in-a-conversation
/api-reference/openapi.json get /api/v1/user/conversations/{conversation_id}/messages
# Rename a conversation
Source: https://docs.4minds.ai/api-reference/conversations/rename-a-conversation
/api-reference/openapi.json patch /api/v1/user/conversations/{conversation_id}
# Create a dataset from files and/or URLs
Source: https://docs.4minds.ai/api-reference/datasets/create-a-dataset-from-files-andor-urls
/api-reference/openapi.json post /api/v1/user/dataset
# Delete a dataset (soft by default)
Source: https://docs.4minds.ai/api-reference/datasets/delete-a-dataset-soft-by-default
/api-reference/openapi.json delete /api/v1/user/dataset/{dataset_id}
# Get a dataset
Source: https://docs.4minds.ai/api-reference/datasets/get-a-dataset
/api-reference/openapi.json get /api/v1/user/dataset/{dataset_id}
# List datasets
Source: https://docs.4minds.ai/api-reference/datasets/list-datasets
/api-reference/openapi.json get /api/v1/user/dataset
# Upload files to an existing dataset
Source: https://docs.4minds.ai/api-reference/datasets/upload-files-to-an-existing-dataset
/api-reference/openapi.json post /api/v1/user/dataset/upload
# Check evaluation name availability
Source: https://docs.4minds.ai/api-reference/evaluations/check-evaluation-name-availability
/api-reference/openapi.json get /api/v1/evaluations/check-name
# Create an evaluation
Source: https://docs.4minds.ai/api-reference/evaluations/create-an-evaluation
/api-reference/openapi.json post /api/v1/evaluations
# Delete an evaluation
Source: https://docs.4minds.ai/api-reference/evaluations/delete-an-evaluation
/api-reference/openapi.json delete /api/v1/evaluations/{evaluation_id}
# Get an evaluation
Source: https://docs.4minds.ai/api-reference/evaluations/get-an-evaluation
/api-reference/openapi.json get /api/v1/evaluations/{evaluation_id}
# List evaluations
Source: https://docs.4minds.ai/api-reference/evaluations/list-evaluations
/api-reference/openapi.json get /api/v1/evaluations
# Run a RAGAS evaluation
Source: https://docs.4minds.ai/api-reference/evaluations/run-a-ragas-evaluation
/api-reference/openapi.json post /api/v1/evaluations/{evaluation_id}/run-ragas
# Start an evaluation run
Source: https://docs.4minds.ai/api-reference/evaluations/start-an-evaluation-run
/api-reference/openapi.json post /api/v1/evaluations/{evaluation_id}/start
# Update evaluation metadata
Source: https://docs.4minds.ai/api-reference/evaluations/update-evaluation-metadata
/api-reference/openapi.json put /api/v1/evaluations/{evaluation_id}
# Run an inference query (streaming SSE)
Source: https://docs.4minds.ai/api-reference/inference/run-an-inference-query-streaming-sse
/api-reference/openapi.json post /api/v1/user/inference
# Create a fine-tuned model
Source: https://docs.4minds.ai/api-reference/models/create-a-fine-tuned-model
/api-reference/openapi.json post /api/v1/user/model
# Delete a model
Source: https://docs.4minds.ai/api-reference/models/delete-a-model
/api-reference/openapi.json delete /api/v1/user/model/{model_id}
# Get a model
Source: https://docs.4minds.ai/api-reference/models/get-a-model
/api-reference/openapi.json get /api/v1/user/model/{model_id}
# List models
Source: https://docs.4minds.ai/api-reference/models/list-models
/api-reference/openapi.json get /api/v1/user/model
# AWS Integrations
Source: https://docs.4minds.ai/aws-integrations
Connect 4MINDS to AWS services using IAM Role Federation or Amazon Cognito. One-time AWS setup that works across Amazon S3, SageMaker, Bedrock, and Lake Formation.
This guide covers the shared AWS setup for connecting 4MINDS to any AWS-based integration — **Amazon S3**, **Amazon SageMaker**, **Amazon Bedrock**, and **AWS Lake Formation**. The connection mechanics (IAM Role Federation or Amazon Cognito) are the same across all of them; only the IAM permissions policy changes per integration.
***
## Choosing a Connection Method
4MINDS supports two connection methods for AWS:
| Method | When to use | Security model |
| ------------------------------------- | -------------------------------------------------------------- | ----------------------------------------------------------------------------------- |
| **IAM Role Federation** (recommended) | You want zero long-lived credentials stored in 4MINDS | Temporary credentials minted on-demand via OIDC + AWS STS (1 hour lifetime) |
| **Amazon Cognito** | Your organization already uses Cognito User Pools for identity | Authenticated Cognito identity assumes an IAM role with Bedrock/S3/etc. permissions |
> Some integrations support additional methods (e.g., Bedrock supports API keys). Those are documented in the integration-specific guides.
***
## AWS Setup (shared across all AWS integrations)
You only need to complete this setup **once per AWS account**, regardless of how many 4MINDS integrations (S3, SageMaker, Bedrock, Lake Formation) you plan to use. When adding a new integration later, you simply attach the appropriate permissions policy to the same IAM role or Cognito authenticated role.
### Step 1: Note Your AWS Region
1. Sign in to the [AWS Management Console](https://console.aws.amazon.com/)
2. The active region appears in the top-right of the AWS console (e.g., `us-east-1`). You'll need to enter this when connecting in 4MINDS
3. Common regions include `us-east-1`, `us-west-2`, `eu-west-1`, `eu-central-1`
### Step 2: Create the IAM Permissions Policy
The policy you attach depends on which 4MINDS integration you're setting up. Skip to the relevant section:
* [Amazon S3 policy](#amazon-s3)
* [Amazon SageMaker policy](#amazon-sagemaker)
* [Amazon Bedrock policy](#amazon-bedrock)
* [AWS Lake Formation policy](#aws-lake-formation)
* [Amazon RDS policy](/rds#rds-iam-permissions-policy)
* [Amazon Redshift policy](/redshift#redshift-iam-permissions-policy)
To create a policy:
1. Go to **AWS Console → IAM** (search for "IAM" in the top search bar)
2. Click **Policies** in the left sidebar
3. Click **Create policy**
4. Click the **JSON** tab (switch from the visual editor)
5. Paste the policy JSON for your integration (see sections below)
6. Click **Next**, name the policy (e.g., `4MINDS-S3-Access`), and click **Create policy**
Keep the policy name handy — you'll attach it in the connection method steps below. You can attach **multiple** 4MINDS policies to the same role if you're connecting to more than one AWS integration.
***
## Connection Method 1: IAM Role Federation (Recommended)
No long-lived credentials are stored. 4MINDS uses OIDC federation to mint short-lived credentials through AWS STS for each request.
### A. Register 4MINDS as an OIDC Identity Provider
1. Go to **AWS Console → IAM → Identity Providers** (left sidebar)
2. Click **Add Provider**
3. Select **OpenID Connect**
4. For **Provider URL**, enter: `https://api.4minds.ai`
5. For **Audience**, enter: `sts.amazonaws.com`
6. Click **Add provider**
You only need to do this once per AWS account.
### B. Create the IAM Role
1. Go to **IAM → Roles** → **Create role**
2. Under **Trusted entity type**, select **Web identity**
3. Under **Identity provider**, select `api.4minds.ai`
4. Under **Audience**, select `sts.amazonaws.com`
5. Click **Next**
6. Attach the permissions policy (or policies) you created in [Step 2](#step-2-create-the-iam-permissions-policy)
7. Click **Next**, enter a **Role name** (e.g., `4MINDS-integration-role`), and click **Create role**
8. **Copy the Role ARN** from the role summary page — it looks like: `arn:aws:iam::123456789012:role/4MINDS-integration-role`
### C. Verify the Trust Policy
1. Open the role you just created
2. Click the **Trust relationships** tab → **Edit trust policy**
3. Confirm it matches this (with your 12-digit AWS account ID):
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam:::oidc-provider/api.4minds.ai"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"api.4minds.ai:aud": "sts.amazonaws.com"
}
}
}
]
}
```
> **Note — External ID is not supported for IAM Role Federation.** AWS STS `AssumeRoleWithWebIdentity` (the API this method uses) does not accept an External ID, so adding an `sts:ExternalId` condition to the trust policy will cause **every connection to fail with `AccessDenied`**. Leave the External ID field blank in 4MINDS for this method.
>
> For additional trust hardening, add a condition on a token claim instead. The JWT 4MINDS mints includes `sub` (e.g. `4minds:org:42:user:7`) and, when applicable, `org_id` and `tenant_id`. For example:
>
> ```json theme={null}
> "Condition": {
> "StringEquals": {
> "api.4minds.ai:aud": "sts.amazonaws.com",
> "api.4minds.ai:sub": "4minds:org:42:user:7"
> }
> }
> ```
### D. Connect in 4MINDS
1. In 4MINDS, open **Integrations** from the main navigation bar and select your integration (e.g., **Amazon S3**)
2. Select the **IAM Role** tab
3. Paste your **IAM Role ARN** (from step B.8)
4. Leave the **External ID** field blank — it is not supported for IAM Role Federation (see the note in [step C](#c-verify-the-trust-policy))
5. Enter your **AWS Region**
6. Fill in any integration-specific fields (see the [Per-Integration Policies & Fields](#per-integration-policies--fields) section)
7. Click **Test Connection**, then **Save Credentials**
***
## Connection Method 2: Amazon Cognito
Use this method if your organization manages AWS access through Amazon Cognito User Pools and Identity Pools.
> **Already have Cognito set up?** Skip to [Gather Your Cognito Details](#gather-your-cognito-details).
### A. Create a Cognito User Pool
1. Go to **AWS Console → Amazon Cognito**
2. Click **Create user pool**
3. Under **Sign-in experience**, check **User name** (and optionally **Email**)
4. Under **Security requirements**, configure your password policy and MFA (select **No MFA** for the simplest setup)
5. Under **Sign-up experience**, uncheck **Enable self-registration**
6. Under **Message delivery**, select **Send email with Cognito**
7. Under **Integrate your app**:
* **User pool name**: `4MINDS-user-pool`
* **App client name**: `4MINDS-app-client`
* **Client secret**: optional — 4MINDS supports both
* Under **Authentication flows**, ensure **ALLOW\_USER\_PASSWORD\_AUTH** is checked (required)
8. Click **Create user pool**
9. **Copy the User Pool ID** — looks like `us-east-1_aBcDeFgHi`
### B. Get the App Client Details
1. In your new User Pool, go to **App integration**
2. Scroll to **App clients and analytics** and open your app client
3. **Copy the Client ID** — looks like `1abc2def3ghi4jkl5mno6pqr`
4. If you generated a client secret, click **Show client secret** and copy it
### C. Create a User in the Pool
1. In your User Pool, go to **Users** → **Create user**
2. Enter a **User name** (e.g., `4MINDS-service-user`)
3. Enter a **Temporary password** or set a permanent one
4. Click **Create user**
If the user status shows `FORCE_CHANGE_PASSWORD`, complete the password change via AWS CLI before connecting in 4MINDS:
```bash theme={null}
aws cognito-idp admin-set-user-password \
--user-pool-id us-east-1_aBcDeFgHi \
--username 4MINDS-service-user \
--password "YourPermanentPassword123!" \
--permanent
```
### D. Create a Cognito Identity Pool
The Identity Pool maps authenticated Cognito users to an IAM role.
1. Go to **Amazon Cognito → Identity pools** → **Create identity pool**
2. Under **User access**, select **Authenticated access**
3. Under **Authenticated identity sources**, select **Amazon Cognito user pool**
4. Under **Configure permissions**, select **Create a new IAM role** and name it (e.g., `4MINDS-cognito-auth-role`)
5. Under **Connect identity providers**, enter the **User Pool ID** and **App Client ID** from steps A and B
6. Enter an **Identity pool name** (e.g., `4MINDS-identity-pool`)
7. Click **Create identity pool**
8. **Copy the Identity Pool ID** — looks like `us-east-1:12345678-abcd-1234-efgh-123456789012`
### E. Attach Permissions to the Cognito Authenticated Role
1. Go to **IAM → Roles** and open the role created in step D.4
2. Click **Add permissions → Attach policies**
3. Attach the permissions policy (or policies) you created in [Step 2](#step-2-create-the-iam-permissions-policy)
### Gather Your Cognito Details
| Field | Where to find it | Example |
| --------------------- | -------------------------------------- | ------------------------------------------------ |
| **User Pool ID** | Cognito → User Pools → Overview | `us-east-1_aBcDeFgHi` |
| **App Client ID** | Cognito → User Pools → App integration | `1abc2def3ghi4jkl5mno6pqr` |
| **App Client Secret** | Same as above (only if generated) | `abcdef123456...` |
| **Identity Pool ID** | Cognito → Identity Pools | `us-east-1:12345678-abcd-1234-efgh-123456789012` |
| **Username** | Cognito user created in step C | `4MINDS-service-user` |
| **Password** | Permanent password for that user | — |
### Connect in 4MINDS
1. Open **Integrations** from the main navigation bar and select your integration
2. Select the **Cognito** tab
3. Fill in all fields from the table above
4. If your app client has a secret, toggle on **App Client Secret** and enter it
5. Enter your **AWS Region** (must match the region of your User Pool and Identity Pool)
6. Fill in any integration-specific fields (see below)
7. Click **Test Connection**, then **Save Credentials**
***
## Per-Integration Policies & Fields
Paste the JSON below into the **JSON** tab when creating a policy in [Step 2](#step-2-create-the-iam-permissions-policy). You can attach multiple policies to the same IAM role or Cognito authenticated role.
### Amazon S3
Connect to Amazon S3 to import datasets and files directly from your S3 buckets.
#### Policy — Broad Access (list all buckets in the account)
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ListAllBuckets",
"Effect": "Allow",
"Action": "s3:ListAllMyBuckets",
"Resource": "*"
},
{
"Sid": "ReadBuckets",
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::*",
"arn:aws:s3:::*/*"
]
}
]
}
```
#### Policy — Least Privilege (restrict to specific buckets)
If you configure **Allowed Buckets** in 4MINDS, you can drop `s3:ListAllMyBuckets` entirely and scope access to just the buckets you expose:
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::my-bucket",
"arn:aws:s3:::my-bucket/*",
"arn:aws:s3:::another-bucket",
"arn:aws:s3:::another-bucket/*"
]
}
]
}
```
| Permission | Purpose |
| --------------------- | --------------------------------------------------------------------- |
| `s3:ListAllMyBuckets` | Enumerates all buckets in the account (skip if using Allowed Buckets) |
| `s3:ListBucket` | Lists objects within a bucket |
| `s3:GetObject` | Downloads file contents during import |
#### Restricting Access with Allowed Buckets
By default, when 4MINDS opens the S3 browser it calls `s3:ListAllMyBuckets` and shows every bucket in your account. If you'd rather expose only a specific set of buckets — for compliance, tenancy isolation, or just to keep the picker tidy — use the **Allowed Buckets** field.
**How it works:**
* With **Allowed Buckets empty**, 4MINDS lists every bucket the role can see. The IAM policy needs `s3:ListAllMyBuckets`.
* With **Allowed Buckets populated**, 4MINDS never calls `ListAllMyBuckets` — it only probes the buckets you named via `s3:ListBucket`. The IAM policy can drop `ListAllMyBuckets` entirely and scope `s3:ListBucket`/`s3:GetObject` to just those bucket ARNs (see [Policy — Least Privilege](#policy--least-privilege-restrict-to-specific-buckets) above).
**Adding buckets in 4MINDS:**
1. Open **Integrations** → **Amazon S3**
2. Scroll to the **Allowed buckets (optional)** section
3. Click **Add bucket** — a new text input row appears
4. Type the **exact bucket name** (e.g., `prod-datasets`), not the ARN and not a path
5. Click **Add bucket** again to add another row; the button is disabled until the current row has a value
6. To remove a bucket, click the trash icon next to its row
7. Click **Test Connection** to verify the role can `head_bucket` on each name, then **Save Credentials**
**Behavior during connection test:**
* Each allowed bucket is probed with `HeadBucket`. If any one fails, the error message names the specific bucket (e.g., *"Access denied on bucket 'prod-datasets'"*) so you can fix a typo or missing grant without hunting.
* Buckets in other regions are reachable but slower — S3 follows a redirect. For best performance, keep all allowed buckets in the region you specified.
**When to use it:**
* **Multi-tenant AWS accounts** where only a subset of buckets contain data you want in 4MINDS
* **Least-privilege IAM setups** where you want to drop `s3:ListAllMyBuckets` and scope `Resource` to specific ARNs
* **Cleaner UX** when your account has hundreds of buckets and users only care about a handful
**When to leave it empty:**
* You want 4MINDS to auto-discover new buckets as they're created without updating the integration
* You already restrict buckets via IAM — in that case, `ListAllMyBuckets` simply returns the subset the role can see
#### 4MINDS Fields
| Field | Required | Notes |
| ------------------- | -------- | -------------------------------------------------------------- |
| **AWS Region** | Yes | Region where your buckets live |
| **Default Bucket** | No | Pre-selected bucket in the browser UI |
| **Allowed Buckets** | No | Restrict which buckets 4MINDS can see. Leave empty to list all |
***
### Amazon SageMaker
Connect to Amazon SageMaker to create 4MINDS models backed by your deployed SageMaker endpoints.
#### Policy
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "SageMakerDiscovery",
"Effect": "Allow",
"Action": [
"sagemaker:ListEndpoints",
"sagemaker:ListModels",
"sagemaker:ListTrainingJobs",
"sagemaker:DescribeEndpoint",
"sagemaker:DescribeEndpointConfig",
"sagemaker:DescribeModel"
],
"Resource": "*"
},
{
"Sid": "SageMakerInvoke",
"Effect": "Allow",
"Action": "sagemaker:InvokeEndpoint",
"Resource": "arn:aws:sagemaker:*:*:endpoint/*"
},
{
"Sid": "IdentityVerification",
"Effect": "Allow",
"Action": "sts:GetCallerIdentity",
"Resource": "*"
}
]
}
```
| Permission | Purpose |
| ------------------------------------------------------------------------- | ------------------------------------------------------- |
| `sagemaker:ListEndpoints` | Discovers deployed inference endpoints |
| `sagemaker:ListModels` | Lists registered SageMaker models |
| `sagemaker:ListTrainingJobs` | Lists training jobs (used for model lineage) |
| `sagemaker:DescribeEndpoint` / `DescribeEndpointConfig` / `DescribeModel` | Retrieves endpoint and model metadata |
| `sagemaker:InvokeEndpoint` | Sends prompts to a SageMaker endpoint at inference time |
| `sts:GetCallerIdentity` | Verifies the connection |
> **Least privilege:** Restrict `sagemaker:InvokeEndpoint` to specific endpoint ARNs if you only want 4MINDS to call certain endpoints. Replace the resource with e.g. `arn:aws:sagemaker:us-east-1:123456789012:endpoint/my-endpoint`.
#### 4MINDS Fields
| Field | Required | Notes |
| -------------- | -------- | ---------------------------------------- |
| **AWS Region** | Yes | Region where your endpoints are deployed |
***
### Amazon Bedrock
Connect to Amazon Bedrock to use foundation models as 4MINDS models.
#### Policy
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "BedrockFoundationModelAccess",
"Effect": "Allow",
"Action": [
"bedrock:ListFoundationModels",
"bedrock:GetFoundationModel",
"bedrock:InvokeModel",
"bedrock:InvokeModelWithResponseStream"
],
"Resource": "*"
},
{
"Sid": "BedrockCustomModelAccess",
"Effect": "Allow",
"Action": [
"bedrock:ListCustomModels",
"bedrock:GetCustomModel"
],
"Resource": "*"
},
{
"Sid": "IdentityVerification",
"Effect": "Allow",
"Action": "sts:GetCallerIdentity",
"Resource": "*"
}
]
}
```
| Permission | Purpose |
| ------------------------------------------------------- | --------------------------------------------- |
| `bedrock:ListFoundationModels` / `GetFoundationModel` | Discovers available foundation models |
| `bedrock:ListCustomModels` / `GetCustomModel` | Discovers fine-tuned custom models (optional) |
| `bedrock:InvokeModel` / `InvokeModelWithResponseStream` | Runs inference at request time |
| `sts:GetCallerIdentity` | Verifies the connection |
> **Least privilege:** Scope `bedrock:InvokeModel` to specific model ARNs. See the [Bedrock guide](/bedrock#least-privilege-restricting-to-specific-models) for examples.
> **Alternative: API Key.** Bedrock also supports authentication via an API key generated in the Bedrock console — no IAM setup needed. See the [Bedrock guide](/bedrock) for details.
#### 4MINDS Fields
| Field | Required | Notes |
| -------------- | -------- | ------------------------------- |
| **AWS Region** | Yes | Region where Bedrock is enabled |
***
### AWS Lake Formation
Connect to Lake Formation to query governed tables via AWS Glue and Athena.
#### Policy
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "GlueCatalogRead",
"Effect": "Allow",
"Action": [
"glue:GetDatabase",
"glue:GetDatabases",
"glue:GetTable",
"glue:GetTables"
],
"Resource": "*"
},
{
"Sid": "AthenaQuery",
"Effect": "Allow",
"Action": [
"athena:StartQueryExecution",
"athena:GetQueryExecution",
"athena:GetQueryResults"
],
"Resource": "*"
},
{
"Sid": "AthenaResultsBucket",
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::",
"arn:aws:s3:::/*"
]
},
{
"Sid": "IdentityVerification",
"Effect": "Allow",
"Action": "sts:GetCallerIdentity",
"Resource": "*"
}
]
}
```
Replace `` with the S3 bucket configured as the Athena query results location (Athena → Settings → Query result location).
| Permission | Purpose |
| ---------------------------------------------------------------------- | ------------------------------------------------------- |
| `glue:GetDatabase` / `GetDatabases` / `GetTable` / `GetTables` | Browses the Glue Data Catalog that backs Lake Formation |
| `athena:StartQueryExecution` / `GetQueryExecution` / `GetQueryResults` | Runs SELECT queries against governed tables |
| `s3:GetObject` / `PutObject` / `ListBucket` on the results bucket | Reads Athena query results written to S3 |
| `sts:GetCallerIdentity` | Verifies the connection |
> **Lake Formation grants:** The IAM policy above allows the **API calls**. You must also grant the role `SELECT` access on the specific databases/tables in the **Lake Formation → Data permissions** console. Without Lake Formation grants, Athena queries will return no rows.
#### 4MINDS Fields
| Field | Required | Notes |
| --------------------------- | -------- | ----------------------------------------------------------------------------------- |
| **AWS Region** | Yes | Region of the Glue Catalog / Lake Formation |
| **Athena Workgroup** | No | Workgroup used for queries (defaults to `primary`) |
| **Athena Results Location** | Yes | `s3://your-athena-results-bucket/path/` — must match the bucket in the policy above |
***
## Testing Your Connection
After saving credentials in 4MINDS:
1. Click **Test Connection** — validates credentials and permissions
2. Success messages vary by integration (e.g., *"Found 12 bucket(s)"* for S3, *"Found 8 foundation model(s)"* for Bedrock)
3. Click **Save Credentials** to persist
***
## Troubleshooting
| Issue | Solution |
| -------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `AccessDenied` on connection test | The IAM policy isn't attached to the role, or the policy is missing a required action |
| `AccessDenied` on a specific bucket (S3) | The bucket is in Allowed Buckets but the policy doesn't include it in its `Resource` — add the bucket ARN or remove it from Allowed Buckets |
| `NoSuchBucket` | The bucket name in Allowed Buckets is a typo or lives in a different region |
| `Unrecognized client` | Wrong AWS region, or the service isn't enabled in that region |
| Role federation fails | Verify the OIDC provider URL is exactly `https://api.4minds.ai` with audience `sts.amazonaws.com` |
| Role federation: `AccessDenied` despite correct provider | Remove any `sts:ExternalId` condition from the trust policy — it is not supported by `AssumeRoleWithWebIdentity` and blocks every connection. Use a `sub`/`tenant_id` claim condition instead |
| Cognito: `FORCE_CHANGE_PASSWORD` | Complete the password change via AWS CLI (see [step C](#c-create-a-user-in-the-pool)) |
| Cognito: `NotAuthorizedException` | Username or password is wrong, or the app client doesn't have `ALLOW_USER_PASSWORD_AUTH` enabled |
| Lake Formation: queries return no rows | Grant the role `SELECT` in **Lake Formation → Data permissions** on the target databases/tables |
***
## Disconnecting
To remove a 4MINDS integration connection:
1. Open **Integrations**, select the integration
2. Click **Disconnect**
This removes stored credentials from 4MINDS. Your AWS resources (IAM roles, OIDC providers, Cognito pools, policies) are not affected — delete them in the AWS Console if no longer needed.
# AWS Marketplace: Helm Deployment
Source: https://docs.4minds.ai/aws-marketplace
Deploy the 4MINDS AI Platform to your own Kubernetes cluster with a single Helm chart, delivered via AWS Marketplace.
Deploy the 4MINDS platform to your own Kubernetes cluster with a single
`helm install`. The umbrella chart carries every service and datastore, and its
own hooks do all in-cluster orchestration (OpenBao init/unseal, DB migrations,
Kafka topics), so no external installer binary is required.
You configure the deployment through one values file: copy the
[values template](#values-template) at the bottom of this page to a file of your
choice (this guide calls it `my-values.yaml`, but the name is arbitrary) and
fill in your hostname, inference endpoints, and options.
The prerequisite commands below use Amazon EKS as the worked example
(`eksctl`, EBS CSI, IRSA). On another Kubernetes distribution, substitute the
equivalent steps: a default StorageClass, a CSI driver, an ingress controller,
and (for KMS auto-unseal) a cloud KMS key + workload identity.
## What you'll do
1. **Prerequisites (Steps 0-7):** confirm your IAM permissions (Step 0), then
prepare the cluster and platform plumbing the chart does *not* create
(cluster/OIDC, storage, ingress, namespace, TLS, image pull, and, only for
production KMS unseal, a KMS key + IAM role).
2. **Configure:** copy the [values template](#values-template) from this page into your own values file and fill it in.
3. **Install:** log in to the Marketplace ECR, pull the chart, and `helm install` with your values.
4. **Verify:** confirm pods are Running and the UI answers.
Sections after that cover upgrades, uninstall, and troubleshooting.
## Before you start
**Required CLI tools.** Install any that are missing before you begin (each link
points to its official install guide):
* [`aws`](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html): the AWS CLI. Authenticate it to the account that owns the cluster.
* [`kubectl`](https://kubernetes.io/docs/tasks/tools/): the Kubernetes CLI.
* [`helm`](https://helm.sh/docs/intro/install/): v3.8+ (OCI registry support required).
* [`eksctl`](https://eksctl.io/installation/): only for the Amazon EKS example steps.
* [`openssl`](https://www.openssl.org/source/): only if you want a self-signed test cert.
Verify each is on your `PATH` with ` version` (e.g. `aws --version`,
`helm version`) before continuing.
**Set these environment variables.** Every command below references them:
```bash theme={null}
export CLUSTER= # e.g. 4minds-prod
export REGION= # e.g. us-east-1
export HOSTNAME_FQDN= # e.g. 4minds.your-company.com
export NAMESPACE=4minds
export ACCOUNT=$(aws sts get-caller-identity --query Account --output text)
```
Then confirm none are empty:
```bash theme={null}
echo "$CLUSTER $REGION $HOSTNAME_FQDN $NAMESPACE $ACCOUNT"
```
**Copy-paste tip.** If a multi-line command (with a trailing `\`) breaks in
your shell, paste it as a single line instead.
## Prerequisites
### Step 0: IAM permissions for the deployer
**What this does:** confirms the IAM identity (user or role) you run these steps
as has every permission the prerequisites need, so you never hit an
`AccessDenied` mid-deployment. Sort this out before anything else.
Across the steps below you (or `eksctl` on your behalf) create an EKS cluster,
CloudFormation stacks, IAM roles/policies, an OIDC provider, EC2/networking
resources, an EBS CSI addon, a KMS key, and you authenticate to the Marketplace
ECR. That is a broad, privileged set of actions.
**Simplest path (recommended for a one-time deploy):** run as an identity with
the AWS-managed policies below attached. This is the least-friction option and is
what most first deployments use:
| Managed policy | Covers |
| ------------------------------------ | -------------------------------------------------------- |
| `AmazonEKSClusterPolicy` | EKS cluster operations |
| `AmazonEC2FullAccess` | VPC/subnets/security groups/instances for the node group |
| `AWSCloudFormationFullAccess` | the stacks `eksctl` creates for the cluster and IRSA |
| `IAMFullAccess` | create the OIDC provider, IRSA roles, and the KMS policy |
| `AWSKeyManagementServicePowerUser` | create the KMS key + alias (KMS unseal, Step 7 only) |
| `AmazonEC2ContainerRegistryReadOnly` | `helm registry login` + pull from the Marketplace ECR |
`eksctl` drives everything through CloudFormation, which in turn creates IAM
roles. There is no smaller managed policy that covers cluster creation
end-to-end. This is why broad IAM/CloudFormation access is needed for the
initial provisioning.
**Least-privilege path (locked-down orgs):** if org policy forbids the broad
managed policies, attach a custom policy granting exactly these actions. Scope
`Resource` down to your account/cluster where you can:
| Service actions | Used by |
| --------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------- |
| `sts:GetCallerIdentity` | reading `$ACCOUNT` (Before you start) |
| `eks:CreateCluster`, `eks:DescribeCluster`, `eks:CreateAddon`, `eks:DescribeAddon`, `eks:CreateNodegroup`, `eks:*` | Steps 1-3 (cluster, OIDC, EBS addon) |
| `cloudformation:CreateStack`, `cloudformation:DescribeStacks`, `cloudformation:*` | every `eksctl` command |
| `ec2:*` (or scoped VPC/subnet/SG/instance/EIP actions) | Step 1 node group + networking |
| `iam:CreateOpenIDConnectProvider`, `iam:CreateRole`, `iam:CreatePolicy`, `iam:AttachRolePolicy`, `iam:PassRole`, `iam:GetRole`, `iam:TagRole` | Steps 1, 2, 7 (OIDC, IRSA roles, KMS policy) |
| `kms:CreateKey`, `kms:CreateAlias` | Step 7 (KMS auto-unseal; skip if `seal.mode: lab`) |
| `ecr:GetAuthorizationToken`, `ecr:BatchGetImage`, `ecr:GetDownloadUrlForLayer`, `ecr:BatchCheckLayerAvailability` | Install (Marketplace ECR login + `helm pull`) |
| `autoscaling:*` | node group scaling resources `eksctl` provisions |
**Marketplace subscription is separate from IAM.** Before pulling the chart,
the account must be subscribed to the 4MINDS product on AWS Marketplace.
Managing that subscription needs `aws-marketplace:Subscribe` /
`aws-marketplace:ViewSubscriptions` (or Marketplace console access) — an
account/billing permission, not something the deploy identity uses at runtime.
**`kubectl` / `helm` permissions are Kubernetes RBAC, not IAM.** As the cluster
creator you are automatically `system:masters` (cluster-admin), so every
`kubectl` and `helm` step in this guide just works. If someone *else* deploys
into a cluster they did not create, grant them cluster-admin (or map their IAM
identity in the `aws-auth` configmap / an EKS access entry).
### Step 1: EKS cluster with OIDC
**What this does:** creates (or reuses) the Kubernetes cluster. `--with-oidc` is
the only hard requirement; IRSA (used for KMS auto-unseal and the EBS driver)
needs it.
Skip the `create cluster` command if you already have a cluster; use your own
version / instance type / node count.
```bash theme={null}
export K8S_VERSION= # e.g. 1.31
export NODE_TYPE= # e.g. m5.2xlarge
export NODE_COUNT= # e.g. 3
eksctl create cluster --name "$CLUSTER" --region "$REGION" \
--version "$K8S_VERSION" --node-type "$NODE_TYPE" --nodes "$NODE_COUNT" --with-oidc
aws eks update-kubeconfig --name "$CLUSTER" --region "$REGION"
kubectl get nodes # all should be Ready
```
**If the cluster already exists,** just make sure the OIDC provider is
associated:
```bash theme={null}
eksctl utils associate-iam-oidc-provider --cluster "$CLUSTER" --region "$REGION" --approve
```
**Sizing (guidance, not a requirement).** The full platform runs many services
plus stateful datastores. A reasonable starting point for the application tier
is **3 × m5.2xlarge** (8 vCPU / 32 GiB each); scale to your workload.
**Inference is separate.** This chart does NOT run models; it points at
OpenAI-compatible endpoints you set in `my-values.yaml` (`llm.*`,
`mlai.embedding`, `symi-gateway.config`, `wren-ai`). Those can be a managed
service or GPU nodes in this same cluster (add a `g5`/`p4` node group and point
the endpoints at the in-cluster services).
**Models and GPU sizing.** The endpoints above serve the models the platform
uses. A typical deployment runs these:
| Model | Role | Configured via |
| ----------------------------- | -------------------------------------------------------------------- | ------------------------------------- |
| GPT-OSS-120B | General LLM (chat history, summarization, wren-ai text-to-SQL, mlai) | `llm.*`, `wren-ai.config.llmEndpoint` |
| Qwen 3.6 35B FP8 | SYMI assistant | `symi-gateway.config.llm` |
| BAAI/bge-m3 | Embeddings | `mlai.embedding` |
| MS-MARCO cross-encoder | Reranker | `llm.crossEncoderEndpoint` |
| numind/NuMarkdown-8B-Thinking | Vision OCR | `mlai.visionOcr` |
| Qwen/Qwen2.5-VL-32B-Instruct | Vision-language (VLM) | `mlai.vlm` |
GPU memory is dominated by the two largest models: GPT-OSS-120B (a sparse
MoE model, \~63 GB of weights) and Qwen 3.6 35B FP8. The rest are small by
comparison. To serve \~100 concurrent users, plan for 2 × NVIDIA RTX PRO
6000 (96 GB each) for inference without fine-tuning. If fine-tuning workloads
must run alongside inference, the requirement increases to 3 × 96 GB RTX PRO
6000\.
### Step 2: Default StorageClass + EBS CSI driver
**What this does:** gives stateful services (PostgreSQL, Redis, Kafka, MinIO,
Qdrant, OpenBao, mlai) a default StorageClass backed by a working CSI driver.
OpenBao persists its vault data on a 2Gi PVC, so this is required even for
`seal.mode: lab`: without persistence, a pod restart re-initializes the vault
and drops the seeded secrets.
```bash theme={null}
# 2a. Mark gp2 (or gp3) as the default StorageClass.
kubectl annotate storageclass gp2 \
storageclass.kubernetes.io/is-default-class=true --overwrite
kubectl get storageclass # exactly one shows "(default)"
# 2b. Install the EBS CSI driver (IRSA role + managed addon).
eksctl create iamserviceaccount --name ebs-csi-controller-sa --namespace kube-system \
--cluster "$CLUSTER" --region "$REGION" --role-name "${CLUSTER}-ebs-csi" \
--attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy \
--approve --role-only
eksctl create addon --name aws-ebs-csi-driver --cluster "$CLUSTER" --region "$REGION" \
--service-account-role-arn "arn:aws:iam::${ACCOUNT}:role/${CLUSTER}-ebs-csi" --force
kubectl -n kube-system rollout status deploy/ebs-csi-controller --timeout=180s
```
### Step 3: ingress-nginx controller
**What this does:** installs the NGINX ingress controller. The chart creates an
`Ingress` of class `nginx`; the controller itself is a prerequisite.
```bash theme={null}
# 3a. Install the controller (as a LoadBalancer).
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm install ingress-nginx ingress-nginx/ingress-nginx \
--namespace ingress-nginx --create-namespace \
--set controller.service.type=LoadBalancer
kubectl -n ingress-nginx rollout status deploy/ingress-nginx-controller --timeout=180s
# 3b. Read the load-balancer hostname, then point your DNS record for
# $HOSTNAME_FQDN at it (CNAME). For a quick local test you can instead map
# the LB's IP to $HOSTNAME_FQDN in /etc/hosts.
kubectl -n ingress-nginx get svc ingress-nginx-controller \
-o jsonpath='{.status.loadBalancer.ingress[0].hostname}'; echo
```
### Step 4: Namespace
**What this does:** creates the namespace everything installs into.
```bash theme={null}
kubectl create namespace "$NAMESPACE"
```
### Step 5: TLS secret for your hostname
**What this does:** provides the certificate the Ingress uses to terminate TLS.
The Ingress reads the secret named by `frontend-backend.ingress.tlsSecretName`
(default `frontend-tls`). The host and TLS entry derive from `global.hostname`
automatically, so you only create the secret here.
**Option A: you already have a cert** (ACM-issued, Let's Encrypt, etc.):
```bash theme={null}
kubectl -n "$NAMESPACE" create secret tls frontend-tls \
--cert=/path/to/tls.crt --key=/path/to/tls.key
```
**Option B: self-signed** (testing/PoC only; browsers will warn):
```bash theme={null}
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
-keyout tls.key -out tls.crt \
-subj "/CN=$HOSTNAME_FQDN" -addext "subjectAltName=DNS:$HOSTNAME_FQDN"
kubectl -n "$NAMESPACE" create secret tls frontend-tls --cert=tls.crt --key=tls.key
```
**Trust the self-signed cert on every machine that opens the UI, otherwise the
app loads but is unusable.**
The 4MINDS frontend opens a WebSocket (`wss://$HOSTNAME_FQDN`) to the
backend. Unlike a normal page load, most browsers will not carry a page-level
"proceed anyway" exception over to a WebSocket: the `wss://` TLS handshake to an
untrusted cert simply fails and the connection is dropped, so the UI renders but
stays disconnected (live updates, chat, and streaming never arrive).
Clicking **Advanced → Proceed** on the page warning is not reliable across
browsers (Safari, in particular, does not reuse that exception for `wss://`).
The only dependable fix is to add the cert to the operating system's trust store
on each client machine, then fully restart the browser.
Add `tls.crt` to the OS trust store on each client machine (restart the browser
afterwards):
**macOS**: add to the System keychain and mark it trusted for SSL:
```bash theme={null}
# imports into the System keychain and marks it always-trusted for SSL
sudo security add-trusted-cert -d -r trustRoot \
-k /Library/Keychains/System.keychain tls.crt
```
Or via **Keychain Access.app**: drag `tls.crt` into the **System** keychain,
double-click it, expand **Trust**, and set **When using this certificate: Always
Trust**.
**Linux** (Debian/Ubuntu):
```bash theme={null}
sudo cp tls.crt /usr/local/share/ca-certificates/4minds.crt
sudo update-ca-certificates
```
On RHEL/Fedora:
```bash theme={null}
sudo cp tls.crt /etc/pki/ca-trust/source/anchors/4minds.crt && sudo update-ca-trust
```
**Windows** (PowerShell as Administrator):
```powershell theme={null}
Import-Certificate -FilePath tls.crt -CertStoreLocation Cert:\LocalMachine\Root
```
Firefox keeps its own trust store. Even after the OS import, add the cert
under **Settings → Privacy & Security → View Certificates → Authorities →
Import**.
For anything beyond a quick PoC, use a real/trusted certificate (Option A: ACM
or Let's Encrypt). With a trusted cert, no client-side trust step is needed at
all and `wss://` works out of the box.
Using a different secret name? Set `frontend-backend.ingress.tlsSecretName`
in `my-values.yaml` to match.
### Step 6: Image pull (usually nothing to do)
**What this does:** lets the cluster pull the images. On AWS Marketplace the
images live in the Marketplace ECR and your EKS node IAM role pulls them
automatically (attach `AmazonEC2ContainerRegistryReadOnly` to the node role if
it isn't already). In that case leave `global.imagePullSecrets: []`.
Only if you mirror the images into your own private registry do you create a
pull secret and list its name under `global.imagePullSecrets`.
### Step 7: OpenBao KMS auto-unseal (production only)
**What this does:** sets up AWS KMS + an IRSA role so OpenBao auto-unseals
without in-cluster keys. Skip this entire step if you use `seal.mode: lab`
(Shamir keys stored in-cluster, fine for test/PoC).
```bash theme={null}
# 7a. Create the KMS key + alias.
KEY_ARN=$(aws kms create-key --description "4minds-openbao-${CLUSTER}" \
--region "$REGION" --query KeyMetadata.Arn --output text)
aws kms create-alias --alias-name "alias/4minds-openbao-${CLUSTER}" \
--target-key-id "$KEY_ARN" --region "$REGION"
# 7b. Create a least-privilege IAM policy scoped to just that key.
aws iam create-policy --policy-name "4minds-openbao-kms-${CLUSTER}" \
--policy-document "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Action\":[\"kms:Encrypt\",\"kms:Decrypt\",\"kms:DescribeKey\",\"kms:GenerateDataKey\"],\"Resource\":\"${KEY_ARN}\"}]}"
# 7c. Create the IRSA ROLE ONLY (--role-only). The chart creates and owns the
# `platform-openbao` ServiceAccount, so do NOT let eksctl create one too.
eksctl create iamserviceaccount \
--name platform-openbao --namespace "$NAMESPACE" \
--cluster "$CLUSTER" --region "$REGION" \
--role-name "4minds-openbao-irsa-${CLUSTER}" \
--attach-policy-arn "arn:aws:iam::${ACCOUNT}:policy/4minds-openbao-kms-${CLUSTER}" \
--role-only --approve
```
Then set these in `my-values.yaml`. The chart binds `roleArn` onto the OpenBao
ServiceAccount as `eks.amazonaws.com/role-arn` automatically (no manual
annotation, survives `helm upgrade`):
```yaml theme={null}
openbao:
seal:
mode: kms
kms:
provider: aws
region:
keyId: alias/4minds-openbao-
roleArn: arn:aws:iam:::role/4minds-openbao-irsa-
```
**Azure / GCP:** set `provider: azure` (`tenantId`/`vaultName`/`keyName`) or
`provider: gcp` (`projectId`/`locationId`/`keyRing`/`cryptoKey`) and bind the
pod identity via `openbao.serviceAccount.annotations` (Workload Identity).
### Prerequisites summary
| # | Prerequisite | Why |
| - | -------------------------------- | ------------------------------------------------------------------- |
| 0 | IAM permissions for the deployer | create the cluster/roles/KMS + pull from ECR without `AccessDenied` |
| 1 | EKS cluster + OIDC | run the platform; OIDC needed for IRSA |
| 2 | Default StorageClass + EBS CSI | stateful services (PVCs) |
| 3 | ingress-nginx controller | serves the Ingress the chart creates |
| 4 | Namespace `4minds` | where everything installs |
| 5 | `frontend-tls` secret | TLS termination for your hostname |
| 6 | Node IAM ECR pull | pull the images (usually automatic on Marketplace) |
| 7 | KMS key + IRSA role | OpenBao auto-unseal (only for `seal.mode: kms`) |
## Configure
**What this does:** creates your deployment's values file. Copy the
[values template](#values-template) from the bottom of this page into a file
(this guide calls it `my-values.yaml`, but the name is arbitrary) and fill in the
required fields: hostname, email, LLM/embedding endpoints, SYMI, wren-ai, seal
mode, and any SSO/integrations you use. Everything else has a sensible default.
You don't need the chart on disk yet; you pull it in the [Install](#install)
step. Just save your filled-in values file somewhere and pass its path to
`helm install` with `-f`.
Use the [values template on this page](#values-template), **not** the
`values-customer-template.yaml` bundled inside the pulled chart. The bundled file
is out of date; the template on this page is the current, supported one.
### (Optional) External S3 instead of the bundled MinIO
By default object storage is the in-cluster MinIO and you set nothing. To use
AWS S3 (or any S3-compatible endpoint) instead, add to `my-values.yaml`:
```yaml theme={null}
storage:
endpoint: "https://s3.us-east-1.amazonaws.com" # empty = bundled MinIO
region: us-east-1
bucketName: 4minds-uploads
secrets: # keep ONE secrets: block; merge with OAuth secrets
s3AccessKey: ""
s3SecretKey: ""
```
Leave `storage.endpoint` / `secrets.s3*` empty to keep the bundled MinIO. If you
also set OAuth client secrets, put the S3 keys under the same `secrets:`
block; a second `secrets:` key silently overrides the first (YAML has no merge).
For Azure Blob, set `storage.useAzure: "true"` and `secrets.azureStorageAccountKey`
in that same block.
## Install
**What this does:** pulls the chart from the AWS Marketplace ECR and deploys the
platform with your values. The chart's hooks then run automatically (OpenBao
init+unseal, backend-secret seeding, Kafka topics, the mlai schema migration).
No manual steps.
**1. Log in to the Marketplace ECR.** After subscribing to the product on AWS
Marketplace, authenticate Helm to the Marketplace registry:
```bash theme={null}
aws ecr get-login-password --region "$REGION" \
| helm registry login --username AWS --password-stdin \
709825985650.dkr.ecr.us-east-1.amazonaws.com
```
The Marketplace ECR host is fixed at `709825985650.dkr.ecr.us-east-1.amazonaws.com`
regardless of `$REGION` — always use `us-east-1` in that URL. `--region "$REGION"`
above only tells the AWS CLI where to fetch the auth token.
**2. Pull and unpack the chart.** Download the chart into an empty directory and
extract it:
```bash theme={null}
mkdir awsmp-chart && cd awsmp-chart
helm pull oci://709825985650.dkr.ecr.us-east-1.amazonaws.com/4minds-ai/4minds-chart --version 3.3.0
tar xf $(pwd)/* && find $(pwd) -maxdepth 1 -type f -delete # extract, then remove the .tgz
```
**3. Install with your values.** Point `-f` at the `my-values.yaml` you filled in
during [Configure](#configure) (use its full path if it lives outside this
directory). The namespace was already created in [Step 4](#step-4-namespace), so
this uses `--namespace` (not `--create-namespace`):
```bash theme={null}
helm install 4minds ./* \
--namespace "$NAMESPACE" \
--set global.awsmpServiceAccountName=backend-service \
-f /path/to/my-values.yaml
kubectl -n "$NAMESPACE" get pods -w # watch until all are Running/Completed
```
The `--set global.awsmpServiceAccountName=backend-service` flag is required for
AWS Marketplace metered billing. Keep it **and** pass `-f my-values.yaml`;
without your values file the platform comes up unconfigured (no hostname, no
inference endpoints).
## Verify
**What this does:** confirms the platform is up and serving.
```bash theme={null}
kubectl -n "$NAMESPACE" get pods # all 1/1 Running
kubectl -n "$NAMESPACE" get ingress 4minds-ingress # ADDRESS = your LB
curl -k https://$HOSTNAME_FQDN/ # returns the 4MINDS frontend
```
Then open `https://$HOSTNAME_FQDN` and sign up (email + password works out of the
box; SSO only if you configured a provider).
## Upgrades
To upgrade, pull the newer chart version the same way as in [Install](#install)
(log in to the ECR, `helm pull` with the new `--version`, unpack), then run the
upgrade from the unpacked chart directory with the same flags:
```bash theme={null}
helm upgrade 4minds ./* \
--namespace "$NAMESPACE" \
--set global.awsmpServiceAccountName=backend-service \
-f /path/to/my-values.yaml
```
Hooks are idempotent (OpenBao "already initialized", mlai migration re-verify).
Auto-generated secrets (PG/Redis/MinIO/Fernet/Tier/SYMI) are preserved across
upgrades via `helm.sh/resource-policy: keep`.
## Uninstall
```bash theme={null}
helm uninstall 4minds -n "$NAMESPACE"
# PVCs and the generated-secrets Secret are retained by design. To wipe fully:
kubectl -n "$NAMESPACE" delete pvc --all
# platform-openbao-keys only exists in seal.mode: lab (Shamir); in kms mode there
# are no unseal keys to delete, so --ignore-not-found keeps this harmless.
kubectl -n "$NAMESPACE" delete secret 4minds-generated-secrets platform-openbao-keys --ignore-not-found
```
Deleting PVCs destroys all data (Postgres, Qdrant, MinIO, etc.).
## Troubleshooting
| Symptom | Likely cause / fix |
| ----------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| Pods `Pending` on PVC | No default StorageClass / EBS CSI not ready (Step 2) |
| `ImagePullBackOff` | Node IAM lacks ECR pull, or wrong `imagePullSecrets` (Step 6) |
| Ingress has no ADDRESS | ingress-nginx controller not installed (Step 3) |
| App pods `CrashLoopBackOff` early on | OpenBao not unsealed yet; `kubectl logs job/4minds-openbao-bootstrap` |
| SYMI: "No API key for provider …" | `symi-gateway.config.llm.baseUrl` + `llmModel` not set → set your SYMI LLM endpoint |
| SYMI UI: "Backend service is unreachable" | leave `symi-gateway.config.apiKey` empty so it shares the backend's generated key |
| Dataset stuck "processing" until refresh | check `kubectl logs deploy/postgres-kafka-bridge` (the `wait-for-tables` initContainer gates trigger install) |
## Values template
This is the current, supported values template; use this one, not the
`values-customer-template.yaml` inside the pulled chart (that bundled copy is
out of date). Copy it into your own values file, fill in the required fields
(hostname, email, inference endpoints), and pass it to `helm install` with `-f`.
```yaml my-values.yaml theme={null}
# =============================================================================
# 4MINDS Platform: Customer Values (helm-native install template)
#
# helm install 4minds ./* --namespace 4minds \
# --set global.awsmpServiceAccountName=backend-service \
# -f /path/to/my-values.yaml (run from the unpacked chart dir)
#
# Fill in the fields below for YOUR deployment. Everything not listed here has a
# sensible default in the chart; you only override what is specific to you.
#
# LEGEND: [REQUIRED] must set · [OPTIONAL] set only if you use that feature
#
# Cluster prerequisites (create BEFORE helm install; the chart does not).
# See INSTALL-GUIDE.md "Prerequisites checklist" (steps 1-7) for the commands:
# • EKS cluster with OIDC (needed for IRSA)
# • A default StorageClass + EBS CSI driver (EKS gp2/gp3, etc.)
# • ingress-nginx controller installed
# • Namespace 4minds (kubectl create namespace 4minds)
# • Secret frontend-tls (kubernetes.io/tls) for your hostname
# • Image pull: on AWS Marketplace the EKS node IAM role pulls from the
# Marketplace ECR, no pull secret needed. Only create one if you mirror
# images into your own private registry (see global.imagePullSecrets).
# • (kms seal only) A KMS key + IRSA role; see INSTALL-GUIDE.md prereq 7.
# =============================================================================
global:
# [REQUIRED] The public DNS name your users reach the platform on.
# EVERYTHING derives from this: backend API URL, frontend, websocket (wss),
# symi, CORS origins, cookie domain, and ALL OAuth redirect URIs.
# Set this correctly and you rarely need to touch any other URL below.
hostname: "4minds.your-company.com"
namespace: 4minds
# Image pull auth:
# On AWS Marketplace the images live in the Marketplace ECR and your EKS
# NODES pull them via their IAM role (AmazonEC2ContainerRegistryReadOnly),
# so NO imagePullSecrets needed. Leave this empty (the chart omits the field).
#
# ONLY set this if you mirror the images into your OWN private registry that
# needs a docker-registry pull secret; then create the secret first and list
# its name here:
# kubectl -n 4minds create secret docker-registry my-pull-secret ...
imagePullSecrets: []
# imagePullSecrets:
# - name: my-pull-secret
security:
allowInsecureImages: true
# -----------------------------------------------------------------------------
# Ingress + TLS (nginx). The host and the TLS block are derived automatically
# from global.hostname above; you do NOT repeat the hostname here. You only
# provide the TLS secret: create `frontend-tls` (kubernetes.io/tls) for your
# hostname before installing (see INSTALL-GUIDE.md, prereq 5).
#
# Advanced: to serve extra hosts or use a different secret per host, set an
# explicit `tls:` list here; it overrides the auto-derived single-host block.
# -----------------------------------------------------------------------------
frontend-backend:
ingress:
enabled: true
className: nginx
tlsSecretName: frontend-tls # host + TLS auto-derive from global.hostname
# -----------------------------------------------------------------------------
# OpenBao seal mode: configure it HERE, under `openbao.seal`.
# (The install/bootstrap reads ONLY `openbao.seal.mode`; there is no separate
# top-level `seal:` to keep in sync.)
# lab = Shamir keys, auto-generated & stored in-cluster (simplest)
# kms = cloud KMS auto-unseal (recommended on AWS/GCP/Azure via IRSA/WI)
# -----------------------------------------------------------------------------
openbao:
seal:
mode: lab # lab | kms
# [OPTIONAL] Fill only when mode: kms. Set the block for YOUR cloud; the
# chart renders the matching seal stanza and (AWS) wires IRSA for you.
# kms:
# provider: aws # aws | azure | gcp
# # --- AWS: KMS key + IRSA role (recommended on EKS) ---
# region: us-east-1
# keyId: alias/4minds-openbao
# roleArn: arn:aws:iam:::role/4minds-openbao-irsa # chart annotates
# # the OpenBao SA
# # --- Azure Key Vault: tenantId + vaultName + keyName ---
# # --- GCP KMS: projectId + locationId + keyRing + cryptoKey ---
# OpenBao data is persisted to a 2Gi PVC by default (needs a default
# StorageClass, INSTALL-GUIDE prereq 2). Override only if needed:
# storage:
# size: 5Gi
# storageClass: gp3
# -----------------------------------------------------------------------------
# [REQUIRED] Email: transactional email (signup verification, invites, reset).
#
# Choose the provider that fits YOUR company; all three are first-class:
# provider: smtp → any standard SMTP server (Gmail, O365, SES-SMTP, ...)
# provider: sendgrid → SendGrid API
# provider: azure → Azure Communication Services
#
# 1) set `provider` to your choice
# 2) fill the SAME-named block below (leave the others empty/removed)
# 3) set senderEmail / senderName
#
# If you do NOT want email verification at all (e.g. internal PoC), set
# disableVerification: "true" and you can leave the provider blocks empty.
# -----------------------------------------------------------------------------
email:
provider: "smtp" # smtp | sendgrid | azure
senderEmail: "noreply@your-company.com"
senderName: "Your Company"
disableVerification: "false" # "true" to skip signup email verification
# --- Option A: SMTP (standard mail server) ---
smtp:
host: "smtp.your-company.com"
port: "587"
user: "noreply@your-company.com"
password: ""
useTls: "true"
# --- Option B: SendGrid ---
sendgrid:
apiKey: ""
# --- Option C: Azure Communication Services ---
azure:
connectionString: ""
# -----------------------------------------------------------------------------
# [REQUIRED] Embedding endpoint (your BGE-M3 / embedding server). mlai uses it.
# -----------------------------------------------------------------------------
mlai:
sslVerifyCertificates: "true" # set "false" only for self-signed inference endpoints
embedding:
endpointUrl: "https://embeddings.your-company.com"
modelName: "BAAI/bge-m3"
dimension: "1024"
# [OPTIONAL] Vision OCR / Vision-Language / image-gen endpoints
visionOcr:
enabled: "true"
endpoint: "https://vlm.your-company.com/v1/chat/completions"
vlm:
enabled: "true"
endpoint: "https://vlm.your-company.com/v1/chat/completions"
flux:
apiUrl: "" # leave empty to disable image generation
# -----------------------------------------------------------------------------
# [REQUIRED] LLM endpoints (your inference stack: vLLM / any OpenAI-compatible)
# -----------------------------------------------------------------------------
llm:
historyEndpoint: "https://llm.your-company.com/v1/chat/completions"
crossEncoderEndpoint: "https://reranker.your-company.com/rerank"
summarizationEndpoint: "https://llm.your-company.com/v1/completions"
models:
# Map model keys to your served endpoints. gpt_oss_120b is used broadly
# (mlai watchdog, finetuning registry, wren-ai text-to-SQL), so set it.
# Unused ones can be left "".
gpt_oss_120b: "https://llm.your-company.com/v1"
gpt_oss_20b: ""
qwen25_72b: ""
qwen3_30b: ""
qwen3_14b: ""
gemma3_12b: ""
nemotron_3_nano: ""
llama4_scout: ""
minimax_m2: ""
# -----------------------------------------------------------------------------
# [REQUIRED] Symi gateway LLM (the assistant). Works with ANY OpenAI-compatible
# endpoint (vLLM, your own gateway, etc.).
# baseUrl - your OpenAI-compatible chat endpoint (REQUIRED for Symi)
# llmModel - the model id served there
# llmProvider - a free-form internal label (used as "/");
# leave it or set to anything, e.g. "openai", "vllm". Not a vendor.
# apiKey: leave empty → Symi shares the backend's auto-generated SYMI_API_KEY.
# -----------------------------------------------------------------------------
symi-gateway:
config:
apiKey: ""
llm:
baseUrl: "https://llm.your-company.com/v1"
llmModel: "your-model"
llmProvider: "internal"
# -----------------------------------------------------------------------------
# [REQUIRED] Wren-AI (natural-language → SQL for structured/analytics queries).
# It calls an OpenAI-compatible chat endpoint to generate SQL. Point llmEndpoint
# at your LLM (the OpenAI-compatible BASE url; code appends /chat/completions;
# same endpoint as llm.models.gpt_oss_120b). Empty → text-to-SQL returns 503.
# -----------------------------------------------------------------------------
wren-ai:
config:
llmEndpoint: "https://llm.your-company.com/v1"
# -----------------------------------------------------------------------------
# [OPTIONAL] SSO + connector integrations. ALL optional; enable ONLY the ones
# you use. None is required: users can always sign up / sign in with
# email + password.
#
# To enable a provider, UNCOMMENT the `oauth:` block below AND the specific
# provider line(s) you need. Put clientId here; the clientSecret goes in the
# `secrets:` block. redirectUri auto-derives from global.hostname (leave it
# out unless you need a custom one). Register the derived URI in each provider:
# sign-in: https:///auth//callback
# integrations: https:///api/v1/integrations//oauth/callback
#
# IMPORTANT: if you uncomment `oauth:`, you MUST uncomment at least one child
# line under it (a bare `oauth:` with no children is invalid YAML/null). If you
# use NO SSO/integrations, leave this whole block commented; the chart's
# defaults apply and everything works.
# -----------------------------------------------------------------------------
# oauth:
# # --- Sign-in / SSO providers ---
# microsoft: { clientId: "" }
# google: { clientId: "" }
# github: { clientId: "" }
# okta: { clientId: "" }
# # --- Data / connector integrations ---
# sharepoint: { clientId: "", scopes: "", tenant: "" }
# office365: { clientId: "", scopes: "", tenant: "" }
# googleDrive: { clientId: "" }
# gsuite: { clientId: "" }
# bigquery: { clientId: "" }
# vertexai: { clientId: "" }
# hubspot: { clientId: "" }
# slack: { clientId: "" }
# box: { clientId: "" }
# dropbox: { clientId: "" }
# dropboxWrite: { clientId: "" }
# salesforce: { clientId: "" }
# supabase: { clientId: "" }
# servicenow: {} # client_id/secret entered per-user in UI
# gong: { clientId: "", apiBaseUrl: "" }
# databricks: { clientId: "", accountId: "", m2mClientId: "" }
# fabric: { clientId: "" }
# azureBlob: { clientId: "" }
# huggingface: { clientId: "" }
# foundry: { clientId: "", scopes: "", tenant: "" }
# -----------------------------------------------------------------------------
# Bundled datastores. Keep enabled unless you point at external services.
# -----------------------------------------------------------------------------
qdrant:
enabled: true
# -----------------------------------------------------------------------------
# [OPTIONAL] Object storage. By DEFAULT the platform uses the bundled MinIO and
# you set NOTHING here. To use EXTERNAL S3 (AWS S3 or any S3-compatible store),
# uncomment `storage:` below and add s3AccessKey/s3SecretKey to the single
# `secrets:` block further down.
# -----------------------------------------------------------------------------
# storage:
# endpoint: "https://s3.us-east-1.amazonaws.com" # empty = bundled MinIO
# region: us-east-1
# bucketName: 4minds-uploads
# useAzure: "false" # "true" for Azure Blob (set secrets.azureStorageAccountKey)
# -----------------------------------------------------------------------------
# [OPTIONAL] Secrets. ONE block for ALL of them: OAuth client secrets for the
# integrations you enabled, plus external-S3 / Azure storage keys. The
# PG/Redis/MinIO/encryption keys are AUTO-GENERATED by the chart; do NOT set
# them here. For production prefer external-secrets / sealed-secrets over
# plaintext.
#
# IMPORTANT: keep this a SINGLE `secrets:` block. YAML has no key merging; a
# second `secrets:` anywhere in the file silently overrides the first. Uncomment
# `secrets:` + at least one child (a bare `secrets:` is invalid/null YAML).
# -----------------------------------------------------------------------------
# secrets:
# # --- External object storage (only if storage.endpoint / useAzure set above) ---
# s3AccessKey: "" # external S3 access key (empty = bundled MinIO)
# s3SecretKey: "" # external S3 secret key
# azureStorageAccountKey: "" # when storage.useAzure: "true"
# # --- OAuth client secrets (only for the providers you enabled above) ---
# microsoftAuthClientSecret: ""
# googleOauthClientSecret: ""
# githubOauthClientSecret: ""
# oktaOauthClientSecret: ""
# sharepointOauthClientSecret: ""
# office365OauthClientSecret: ""
# hubspotAuthClientSecret: ""
# slackClientSecret: ""
# boxOauthClientSecret: ""
# dropboxClientSecret: ""
# dropboxWriteClientSecret: ""
# bigqueryOauthClientSecret: ""
# vertexaiOauthClientSecret: ""
# salesforceAuthClientSecret: ""
# supabaseAuthClientSecret: ""
# gongOauthClientSecret: ""
# databricksAuthClientSecret: ""
# databricksM2mClientSecret: ""
# fabricAuthClientSecret: ""
# azureBlobOauthClientSecret: ""
# hfOauthClientSecret: ""
# hfApiToken: "" # HuggingFace API token (private datasets)
# -----------------------------------------------------------------------------
# [OPTIONAL] Images. DEFAULT to the official 4MINDS registry (do NOT override
# for a real install). Only set this if you mirror images into your own
# registry. NOTE: this belongs under the SAME top-level `global:` block at the
# top of this file; move it there rather than adding a second `global:` key.
# -----------------------------------------------------------------------------
# global:
# imageRegistry: "your-registry.example.com/4minds"
```
# Amazon Bedrock
Source: https://docs.4minds.ai/bedrock
This guide walks you through connecting your AWS account to 4MINDS so you can use Amazon Bedrock foundation models. There are three connection methods available:
1. **IAM Role Federation** *(recommended)* — no credentials stored; temporary STS credentials minted per request
2. **Bedrock API Key** — simplest setup, generated directly in the Bedrock console
3. **Amazon Cognito** — use if your organization already manages AWS access through Cognito
Methods 1 and 3 share AWS setup with every other 4MINDS AWS integration (S3, SageMaker, Lake Formation). The steps below focus on what's **specific to Bedrock**; for the generic role/Cognito setup, see [AWS Integrations](/aws-integrations).
## Provider prerequisites
Before connecting Bedrock to 4MINDS, complete the following in your AWS account:
* Request access to the foundation models you want to use in **Amazon Bedrock → Model access**.
* Wait for each model's status to show **Access granted**. Some models require AWS approval and may not be available immediately.
* Confirm your IAM role (or Bedrock API key) has `bedrock:InvokeModel` permission for the granted models — this is covered by the [Bedrock IAM Permissions Policy](#bedrock-iam-permissions-policy) below.
See AWS's [Manage access to Amazon Bedrock foundation models](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html) for the full process.
***
## Bedrock IAM Permissions Policy
Whichever IAM-based method you pick (IAM Role Federation or Cognito), you'll attach this policy to the role:
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "BedrockFoundationModelAccess",
"Effect": "Allow",
"Action": [
"bedrock:ListFoundationModels",
"bedrock:GetFoundationModel",
"bedrock:InvokeModel",
"bedrock:InvokeModelWithResponseStream"
],
"Resource": "*"
},
{
"Sid": "BedrockCustomModelAccess",
"Effect": "Allow",
"Action": [
"bedrock:ListCustomModels",
"bedrock:GetCustomModel"
],
"Resource": "*"
},
{
"Sid": "IdentityVerification",
"Effect": "Allow",
"Action": "sts:GetCallerIdentity",
"Resource": "*"
}
]
}
```
**What each permission does:**
| Permission | Purpose |
| --------------------------------------- | ---------------------------------------------------------------- |
| `bedrock:ListFoundationModels` | Lists available foundation models (Claude, Llama, Mistral, etc.) |
| `bedrock:GetFoundationModel` | Retrieves details about a specific foundation model |
| `bedrock:InvokeModel` | Sends prompts and receives responses from models |
| `bedrock:InvokeModelWithResponseStream` | Enables streaming responses for real-time output |
| `bedrock:ListCustomModels` | Lists custom fine-tuned models in your account |
| `bedrock:GetCustomModel` | Retrieves details about a specific custom model |
| `sts:GetCallerIdentity` | Verifies the connection is authenticated correctly |
Name the policy something memorable like `4MINDS-Bedrock-Access` — you'll reference it when attaching permissions in the role/Cognito setup.
### Least-Privilege: Restricting to Specific Models
The policy above uses `"Resource": "*"` for broad access. To scope to specific regions or models:
**Restrict to a single region:**
```text theme={null}
arn:aws:bedrock:us-east-1::foundation-model/*
```
**Restrict to specific models:**
```json theme={null}
"Resource": [
"arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-sonnet-20240229-v1:0",
"arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-haiku-20240307-v1:0",
"arn:aws:bedrock:us-east-1::foundation-model/meta.llama3-70b-instruct-v1:0"
]
```
The ARN format is: `arn:aws:bedrock:::foundation-model/`
> **Note:** `ListFoundationModels` and `ListCustomModels` still require `"Resource": "*"`. Split these into a separate statement if you scope `InvokeModel` to specific model ARNs.
***
## Connection Methods
### Method 1: IAM Role Federation (Recommended)
Follow the full role federation setup in **[AWS Integrations → IAM Role Federation](/aws-integrations#connection-method-1-iam-role-federation-recommended)**. Attach the `4MINDS-Bedrock-Access` policy (from [above](#bedrock-iam-permissions-policy)) when creating the IAM role.
Then, in 4MINDS:
1. Open **Integrations** from the main navigation bar and select **Amazon Bedrock**
2. Select the **IAM Role** tab
3. Paste your **IAM Role ARN**
4. Leave the **External ID** field blank — it is not supported for IAM Role Federation (see [AWS Integrations → Verify the Trust Policy](/aws-integrations#c-verify-the-trust-policy))
5. Enter your **AWS Region** (must match the region where you've enabled Bedrock model access)
6. Click **Test Connection**, then **Save Credentials**
***
### Method 2: Bedrock API Key
The simplest setup — generate an API key directly from the Bedrock console. No IAM role or Cognito pool needed.
#### AWS Setup
1. Go to **AWS Console → Amazon Bedrock**
2. In the left sidebar, click **API keys**
3. Choose your key type:
* **Short-term API key** (recommended for production) — expires when your console session expires (12 hours). Click **Generate short-term API keys**
* **Long-term API key** — can last longer than 12 hours. Click **Long-term API keys**, configure expiration, and generate
4. **Copy the API key** once generated
> **Note:** Short-term keys require regenerating and updating your 4MINDS connection when they expire. For a set-it-and-forget-it setup, use IAM Role Federation (Method 1).
#### Connect in 4MINDS
1. Open **Integrations** and select **Amazon Bedrock**
2. Select the **API Key** tab
3. Paste the **Bedrock API key**
4. Enter your **AWS Region**
5. Click **Test Connection**, then **Save Credentials**
> **Security note:** API keys are encrypted at rest. For production environments requiring maximum security, prefer IAM Role Federation (Method 1) — it stores nothing long-lived.
***
### Method 3: Amazon Cognito
Follow the full Cognito setup in **[AWS Integrations → Amazon Cognito](/aws-integrations#connection-method-2-amazon-cognito)**. Attach the `4MINDS-Bedrock-Access` policy (from [above](#bedrock-iam-permissions-policy)) to the Cognito authenticated role.
Then, in 4MINDS:
1. Open **Integrations** and select **Amazon Bedrock**
2. Select the **Cognito** tab
3. Fill in the Cognito fields (User Pool ID, App Client ID, App Client Secret if used, Identity Pool ID, Username, Password)
4. Enter your **AWS Region** (must match your User Pool and Identity Pool region)
5. Click **Test Connection**, then **Save Credentials**
***
## After Connecting: Adding Models
Once connected:
1. Open **Integrations** and select **Amazon Bedrock**
2. Browse the list of available foundation models — all supported models appear automatically
3. Click to register models you want to use in your workspace
4. Registered models appear in your model selector for conversations
***
## Supported AWS Regions
Amazon Bedrock is available in select regions. Common options:
* `us-east-1` (N. Virginia)
* `us-east-2` (Ohio)
* `us-west-2` (Oregon)
* `eu-west-1` (Ireland)
* `eu-central-1` (Frankfurt)
* `ap-southeast-1` (Singapore)
* `ap-northeast-1` (Tokyo)
Check the [AWS Regional Services List](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) for current availability.
***
## Troubleshooting
| Issue | Solution |
| --------------------- | --------------------------------------------------------------------------------------------- |
| "No models found" | Verify the region is correct and that your IAM policy includes `bedrock:ListFoundationModels` |
| Connection times out | Verify the region is correct and Bedrock is enabled there |
| "Access Denied" | Confirm the [IAM policy](#bedrock-iam-permissions-policy) is attached to the correct identity |
| "Unrecognized client" | Wrong region, or Bedrock isn't enabled in that region |
| API key expired | Generate a new short-term key or use a long-term key / IAM Role Federation |
For method-specific issues (OIDC trust policy, Cognito password states, etc.), see [AWS Integrations → Troubleshooting](/aws-integrations#troubleshooting).
***
## Disconnecting
1. Open **Integrations** and select **Amazon Bedrock**
2. Click **Disconnect**
This removes stored credentials from 4MINDS. Your AWS resources are not affected.
# Changelog & Release Notes
Source: https://docs.4minds.ai/changelog
## August 2026
### Release 08.06.26 — August 6, 2026
#### Chat & Personas
* ***Structured chat attachments***: CSV, XLSX, Parquet, and JSON files can now be attached directly in chat and queried with natural language
* Improved ***mobile chat usability***: responsive input and menu layouts, updated conversation-menu positioning, and shared styling across authentication pages, including an iOS input-zoom fix
#### Integrations
* Added ***DBT integration***: connect DBT projects via OAuth and sync datasets directly into 4MINDS
#### User Interface Updates
* Added a ***support portal*** link in the profile dropdown that opens the HubSpot customer portal
* Renamed ***Invite Friends*** to ***Recommend 4MINDS***
* Marketing site pricing page now links to the AWS and Azure marketplace listings
#### Under the hood
* Graph visualization stability fixes
* TSV file data ingestion fixes
## July 2026
### Release 07.29.26 — July 29, 2026
#### Authentication
* ***Enterprise SSO self-setup***: Enterprise customers can now configure single sign-on directly from the marketplace signup flow. A new **Set up SSO** tab lets a company's first admin enter their OIDC configuration once, and every user after them signs in with just their email
#### Deployment
* ***Multi-cluster reliability***: marketplace signups now land and stay on the correct cluster for the account
### Release 07.24.26 — July 24, 2026
#### Chat & Personas
* ***Thread-local chat attachments***: attachments are scoped strictly to the thread they are uploaded in and are never written to the Knowledge Graph or shared Knowledge Base, keeping them isolated from other users and other threads
* ***Attachment sources in the response panel***: attached files selected by the planner are now surfaced alongside Knowledge Base citations in the Sources panel and persisted to response metadata
* ***Attachment filenames in conversation history***: planner-selected filenames are appended to the stored query record, enabling accurate semantic retrieval over past turns that involved file attachments (streamed responses are unaffected)
* ***Per-file token budget for large attachment sets***: replaced the previous single-clip strategy with a two-pass allocation that reserves per-file header overhead and distributes the remaining budget evenly, so every attached file is represented in context regardless of attachment count
* ***Web search sources in the Sources panel***: web results the model actually used are now surfaced alongside Knowledge Base and attachment sources for full provenance on web-grounded answers
* ***Chunk ID suppression in responses***: response assembly now prohibits internal chunk identifiers from appearing in user-facing output, preventing raw UUIDs from leaking into the streamed response
#### Integrations
* ***Redshift schema expansion***: removed view-type filtering from Redshift schema introspection so the full schema surface — including views and additional schema objects — is available to Enterprise users with Redshift connectors
#### Datasets
* ***Truthful OCR failure propagation***: extractions where all pages fail OCR now return a typed error instead of an empty document with inflated quality scores; partial extractions report quality proportional to successful pages, and failure markers no longer pollute embeddings
* ***VLM image normalization***: high-resolution images are now normalized to safe dimension and pixel bounds before submission to the Vision Language Model, preventing request timeouts across the OCR extractor and the synchronous and asynchronous processors
* ***Per-attempt VLM retry deadlines***: each VLM retry attempt now receives an independent timeout with capped, interruptible backoff, replacing a shared deadline that caused all retries to fail instantly after the first timeout
#### Thread Deletion
* ***Response-engine data purge***: deleting a conversation thread now triggers a cascading, asynchronous purge across PostgreSQL (conversation record, messages, summaries, attachments), Qdrant (per-tenant conversation embeddings scoped to the deleted thread and its owner), and the hot in-memory cache
* The internal deletion route is idempotent and ownership-enforced: unknown or already-deleted threads are handled gracefully, and foreign-owned data cannot be removed
#### Observability
* ***WebSocket query metrics***: query count and duration are now recorded on the WebSocket path, closing a gap where normal chat traffic produced no signal in Grafana
* ***Per-stage span instrumentation***: full per-stage tracing across orchestrator, embedding, context-assembler, and model-selector stages, with correct lifecycle handling for success, fallback, stream interruption, client cancellation, and disconnect (interrupted streams are excluded from the completed-query counter)
* ***OpenTelemetry off-by-default and crash-proof***: telemetry providers only initialize when an OTLP endpoint is explicitly configured, initialization failures roll back to a disabled state instead of crashing the service, and exports run in background threads with a bounded timeout so they never block the request path. The Helm chart OTEL block is conditionally rendered for deployments that omit OTEL configuration entirely
#### Under the hood
* ***Centralized model ID configuration***: the active model identifier is now an environment-configurable knob with semantic role constants for planning, synthesis, and system default, replacing hardcoded string literals across all call sites. The active model can be changed by updating a single environment variable and restarting the service
* ***Summary embeddings in orchestrator planning***: top document summaries ranked by vector similarity against the current query are now retrieved from the per-tenant summary collection at query time and injected into the orchestrator planning prompt, improving the relevance and precision of the retrieval plan. Includes stability fixes in the summarization-engine storage and API layer
### Release 07.22.26 — July 22, 2026
#### Integrations
* Added ***Azure Database for PostgreSQL***: connect using OAuth or your database credentials
* ***Redshift***: schema filtering improvements for data connections
#### Chat & Personas
* ***Web search sources***: sources used in web-search answers now appear in the Sources panel alongside your documents, with each source shown as a clickable link reflecting only what the answer actually used
* ***Chat attachments***: URLs and integrations are now supported as attachment sources, plus minor fixes and improvements
#### Under the hood
* UI improvements and bug fixes
### Release 07.15.26 — July 15, 2026
#### User Interface Updates
* Refreshed ***look and feel*** across the platform: updated fonts and typography, refined themes, and an improved mobile experience
* Deprecated ***Docs mode*** in chat; a link to [docs.4minds.ai](https://docs.4minds.ai) has been added to the profile icon dropdown
#### Chat & Personas
* Added ***chat attachments***: attach files directly in chat threads (first release — the feature will continue to evolve)
#### Deployment
* ***AWS Marketplace***: the 4MINDS deployment package is now available via AWS Marketplace, with prerequisites and setup documentation
#### Under the hood
* API improvements, stability enhancements, performance optimizations, and bug fixes
***
## June 2026
### Integrations
* Added ***S3 and Dropbox dataset sync***: connect S3 buckets and Dropbox folders directly as synced dataset sources
* Fixed ***dataset sync failures*** ("connection is closed") on slow-responding integrations like OneDrive and Box
* Fixed ***Google Drive sync*** getting stuck in a loop on certain folder structures
* Fixed ***integration file attachments*** not attaching correctly in Advanced Mode
### Datasets
* Added end-to-end support for ***CSV, XLSX, XLS, and Parquet files*** in the Wren/DuckDB structured data pipeline, including GPT-OSS model support
* Added new ***text and markdown accumulating chunkers*** for higher-quality document ingestion
* Increased ***uncompressed ZIP upload limit*** to per-tier settings (up to 2,000 MB), replacing the previous 100 MB cap
* Fixed ***ZIP upload validation*** errors around file size and file count
* Set a ***30-minute minimum dataset sync interval*** to prevent runaway sync loops
* Capped the ***graph visualization*** at 500 files per view to keep large graphs responsive
### Chat & Personas
* Fixed ***custom persona descriptions*** not reaching the model prompt on the REST path
* Fixed ***user messages disappearing*** immediately after send when starting a new conversation
* Smaller, collapsed-by-default ***Sources dropdown*** for a cleaner chat experience
* Fixed ***duplicate files*** appearing in the Sources list
* Fixed ***"Add Data"*** in the right panel not working in fullscreen mode
* Fixed ***dataset share confirmation message*** being lost after sharing
### API
* `/v1/chat/completions` now correctly handles ***array-form message content*** for OpenAI-compatible clients
* OpenAI integration now checks ***deleted-model status*** before running inference
### Admin
* Added a toggle to ***show or hide 4MINDS internal users*** in the Admin usage view
### Authentication
* Fixed an error blocking ***new user signups***
### Legal
* Updated ***Privacy Policy*** content
***
## May 2026
### User Interface Updates
* Refreshed ***login and sign-up pages*** with updated styles (May 1)
### SYMI
* Launched ***SYMI versioning***: save snapshots of your SYMI workspace, view a changelog of what's changed, and reset the workspace back to a clean state with a confirmation prompt before anything is wiped (May 5)
* Added ***PDF renderer*** and ***LibreOffice renderer***: SYMI can now generate PDF files (May 5)
### Datasets
* Added ***`.trs` file support***: users can now add `.trs` files to their datasets (May 13)
***
## April 2026
### Integrations
* Added ***Google Workspace (GSuite) integration*** for access to Gmail, Drive, Calendar, and Docs (April 3)
* Added ***Gong integration*** for revenue intelligence and sales conversation data (April 3)
* Added ***Amazon RDS integration*** for relational database connectivity and structured data access (April 3)
* Added ***BigQuery integration*** for cloud data warehouse access and large-scale analytics (April 3)
### Developer Tools
* Launched ***4MINDS MCP Server*** (April 15)
### User Interface Updates
* Launched ***SYMI AI Assistant*** with a major interface and control center overhaul (April 17)
## March 2026
### Teams
* Launched ***Teams Tab***: organization administrators can now create and manage teams, assign users, and organize access to shared resources based on team membership
* Added ***Shared Resources View***: users can view models and datasets shared with them by other organization members, accessible directly from the Teams tab
* Added ***Granular Resource Sharing***: models and datasets can now be shared with the entire organization, specific teams, or individual users
### Role-Based Access Control (RBAC)
* Launched ***Multi-Layer Role Hierarchy***: roles now operate across three levels — organization (Owner, Admin, Billing Manager, Member, Guest), team (Owner, Admin, Member, Viewer, Guest), and system - with permissions cascading accordingly
* Added ***Access Levels***: users are assigned Basic, Plus, or Admin access tiers based on their subscription plan, controlling feature availability across the platform
* Added ***Custom Roles & Granular Permissions***: admins can grant or revoke individual `resource:action` permissions for any role, scoped at the organization, team, or resource level
* Added ***Resource-Level Access Control***: per-model and per-dataset access grants can be targeted at specific users, teams, or the entire organization — with support for time windows, token limits, rate limits, and usage duration restrictions on model grants
* Added ***RBAC Audit Trail***: all role and permission changes are logged with full context including actor, target, resource, IP address, and timestamp
* Added ***SSO Auto-Provisioning***: SSO groups can now be mapped to teams with a default access level, enabling automatic role assignment on login across Azure AD, Okta, Google, GitHub, and LDAP
***
## February 2026
### Agentic Platform
* Launched ***Agentic AI Platform***: rotating agent status indicators with staggered timing for real-time visibility into agent activity
* Added ***Extended Thinking Display***: "Extending Thinking..." indicator during response regeneration
* Improved ***Sources Experience***: sources now displayed in a dropdown with pagination for cleaner chat interface
### External Model Integrations
* Added ***Amazon SageMaker Integration***: create models on the 4MINDS platform directly from your existing SageMaker models
* Added ***Amazon Bedrock Integration***: connect to AWS-managed foundation model endpoints hosted on Amazon Bedrock
* Added ***Microsoft Foundry Integration***: create models from external models deployed on Microsoft Foundry
### Integrations
* Added ***Databricks Serverless Updates***: OAuth/M2M authentication and Personal Access Token (PAT) support for secure, flexible connectivity
***
## January 2026
### User Interface Updates
* Launched ***Major UI Refresh*** with significant visual and structural improvements across the platform (January 20)
* Added ***AI Assistant Panel***: dedicated left sidebar with contextual onboarding and help system for guided workflows
* Introduced ***Enhanced Model Info Panel***: redesigned right sidebar consolidating Quick Actions, model details, and easy access to evaluations, API keys, and model settings
* Added ***DOC MODE Toggle***: new interface mode for enhanced document-focused workflows
* Implemented ***Three-Column Layout***: improved organization and workflow efficiency across the platform
* Enhanced ***Visual Canvas***: updated graph rendering with force simulation and viewport culling for better performance
* Streamlined ***Evaluations View***: updated evaluations table with improved status indicators and action menus
### Integrations
* Added Supabase integration for database and backend access (January 21)
* Added ServiceNow integration for IT service management workflows (January 21)
* Added Slack integration for team communication and messaging data (January 21)
* Added Google Drive integration for cloud document access (January 21)
* Added Dropbox integration for file storage and sharing (January 21)
* Added Splunk integration for log and security data analysis (January 21)
### Authentication
* Added Single Sign-On (SSO) with AWS (January 21)
* Added Single Sign-On (SSO) with Ping Identity (January 21)
* Added Single Sign-On (SSO) with Okta (January 21)
***
## December 2025
### Integrations
* Added Salesforce integration for CRM access to leads, accounts, contacts, opportunities, and sales analytics (December 9)
* Added HubSpot integration for CRM data, contact management, conversations, and email tracking (December 4)
***
## November 2025
### New Features
* Added ***Log Timeline Chart*** for visual monitoring: view system activity over time with a color-coded timeline displaying log event distribution, severity levels (Debug, Info, Warning, Error, Critical), and activity patterns to quickly identify spikes or gaps before diving into detailed logs (November 25)
* Added ***Model as Judge*** evaluation method: automatically compare your customized model against a base foundation model with ChatGPT acting as an AI judge to evaluate responses side-by-side, providing detailed analysis of factual grounding, key differences, and winner rationale to help identify knowledge gaps and guide training data improvements (November 24)
* Introduced ***System Logging*** for real-time visibility into AI model operations with live log streams, severity-level filtering (DEBUG, INFO, WARNING, ERROR, CRITICAL), search functionality, and log management tools including pause/resume, clear, and download capabilities (November 14)
* Implemented ***Automatic Data Synchronization (Rsync)***: datasets created from integrations now automatically sync on login. The platform tracks a manifest of your source files and fetches any new additions, keeping your AI knowledge base current without manual re-imports (November 13)
### Integrations
* Added Microsoft Fabric integration: browse and import from Lakehouses (files) and Warehouses (tables) for unified analytics platform access (November 25)
* Added Box integration for cloud-stored documents and file management (November 13)
* Added NetApp integration for enterprise storage access (November 12)
* Added Office 365 integration for seamless access to emails, calendars, OneDrive files, Teams messages, and OneNote (November 11)
### Authentication
* Added Single Sign-On (SSO) with Microsoft (November 11)
***
## October 2025
### New Features
* Introduced *Evaluations* functionality to automatically generate comprehensive model performance reports with key metrics, enabling data-driven decisions on model quality (October 31)
### Development Environment Enhancements
* Added Cloud Shell support for browser-based command-line access to 4MINDS (October 31)
### Integrations
* Added support for 5 new integrations: Amazon S3, CoreWeave Storage, SharePoint, Snowflake, Google Cloud Storage (October 28)
* Added dataset and file import capabilities from Databricks, Hugging Face, and Azure Blob Storage (October 17)
* Added credential management for connected integrations (October 16)
### Web Search & Scraping
* Added web scraping feature supporting multiple URL inputs for real-time data extraction (October 15)
* Introduced web search toggle during chat and inference sessions (October 7)
### File Format Support
* Added support for 10 new file types : BMP, DOCX, GIF, JPEG, JPG, MD, PDF, PNG, TIFF, XLSX (October 28)
* Added support for graph exports: SVG, PNG, JSON, GraphML, GEXF, PDF (October 1)
### Coming Soon
* Platform evaluation support, providing insights into usability, features, and performance.
* OCR support for image uploads and dataset integration
* API key generation for custom application integrations
***
# Cloud Shell
Source: https://docs.4minds.ai/cloud-shell
Use the 4MINDS Cloud Shell to interact with your models, datasets, and personas through the command-line interface.
The Cloud Shell provides a powerful command-line interface (CLI) for managing and interacting with your 4MINDS resources. Access it directly from the platform to execute commands, query models, and manage your AI workflows.
## Accessing Cloud Shell
To open Cloud Shell, click the terminal icon in the top right corner of the screen, next to the notifications icon.
![Screen Shot2025 10 31at7 45 40PM Pn]()
## Getting Help
Get help for specific command categories:
```bash theme={null}
help --models # Model management commands
help --datasets # Dataset management commands
help --chat # Chat and conversation commands
help --personas # Persona commands
```
## General Commands
Essential commands for navigating and using the Cloud Shell:
| Command | Description | Shortcut |
| -------- | ----------------------------- | -------- |
| `whoami` | Show current user information | - |
| `api` | Show available API endpoints | - |
| `clear` | Clear terminal screen | Ctrl+L |
| `exit` | Close terminal | Esc |
## Quick Start
Get started quickly with these common commands:
### List your Models
```bash theme={null}
models
```
View all AI models in your account.
### List your Datasets
```bash theme={null}
datasets
```
View all datasets available in your account.
### Chat with a Model
```bash theme={null}
chat
```
Start an interactive chat session with a specific model. Replace `` with your model's identifier and `` with your message.
**Example:**
```bash theme={null}
chat my-chatbot-model "What are the key features of 4MINDS?"
```
## Command Syntax
For detailed command syntax and options, use the help flag with the specific category:
```bash theme={null}
help --
```
Replace `` with `models`, `datasets`, `chat`, or `personas` to see detailed documentation for that command group.
# Manage and Interact with Your Model
Source: https://docs.4minds.ai/control-center
The Control Center serves as your primary workspace for model development and optimization. After creating a model through the setup wizard, select it in the Control Center to begin in-depth work. Unlike other tabs that focus on specific components (personas, datasets, configurations), the Control Center provides a unified view across all model aspects - a single interface for comprehensive model management and iteration.
![Screen Shot2025 10 09at2 03 27PM Pn]()
![Screen Shot2025 10 09at2 03 27PM Pn]()
## AI Assistant Chat Interface
The AI Assistant is an in-platform chat tool that answers questions about your connected knowledge base. Ask in natural language and get structured, sourced answers without leaving the Control Center.
### Interface Layout
The chat interface has three main areas:
* **Response panel** — The AI's answers, formatted with numbered steps, bold key terms, and inline code where applicable.
* **Sources bar** — Indicates how many knowledge sources were referenced for the response.
* **Message input** — Where you type questions and configure response settings.
### Reading Responses
Responses are formatted for clarity and may include:
* **Numbered steps** for sequential or multi-part processes.
* **Bold terms** at the start of each step to highlight key concepts.
* **Inline code** for exact values, commands, or parameters that should be used verbatim.
* **Italic notes** at the end of a response to flag gaps or limitations in the available knowledge base.
A typical response follows this pattern:
1. **Action** — A plain-language explanation of what happens in this step.
2. **Action** — Further detail, including any relevant technical specifics.
If the connected knowledge base does not fully cover a topic, the assistant says so explicitly and indicates where additional detail may be needed.
### Sources
At the bottom of each response, a **Sources** bar shows how many documents were referenced (e.g., `Sources (6)`).
* Click the **Sources** bar to expand and view the individual documents consulted.
* A higher source count generally means the answer is supported by a broader range of material.
### Message Input
The input bar at the bottom of the screen is where you interact with the assistant.
**Toolbar icons:**
| Icon | Function |
| --------- | --------------------------------------------------------------------- |
| **+** | Attach a file or additional content to your message. |
| **Globe** | Enable web search to supplement the knowledge base with live results. |
| **\** | Insert a code block into your message. |
| **Image** | Attach an image to your message. |
**Persona selector:** The **None** dropdown to the right of the toolbar lets you pick a persona, which adjusts the assistant's tone or role. Defaults to **None** (standard assistant behavior). See [Personas](/persona-configuration).
**Send:** Click the **arrow (↑)** button or press **Enter** to submit your message.
### Settings and Modes
Two toggles appear in the bottom bar:
| Toggle | Description |
| ------------ | ------------------------------------------------------------------------------------------------------------ |
| **Advanced** | Enables advanced response options. Toggle on for more detailed or technical outputs. |
| **Doc** | Switches the assistant into documentation mode, optimizing responses for structured, reference-style output. |
### Tips for best results
* **Ask specific questions.** The more precise your query, the more targeted the response.
* **Check the Sources count.** A higher source count generally means a more comprehensive answer.
* **Read italic notes carefully.** They indicate gaps in the knowledge base where you may need additional resources.
* **Use Doc mode** when you want responses formatted for documentation or knowledge-base use.
### Limitations
* The assistant's answers are only as complete as the knowledge base it has been trained on. If a detail isn't covered, the assistant says so explicitly.
* The assistant cannot take actions on your behalf — it provides information only.
### Quick Reference
| I want to… | How |
| ------------------------------------- | --------------------------------------------- |
| Ask a question | Type in the message input and press **Send**. |
| See which sources were used | Click the **Sources** bar to expand. |
| Attach a file to my question | Click the **+** icon in the toolbar. |
| Search the web for additional context | Click the **Globe** icon. |
| Include a code snippet in my question | Click the **\** icon. |
| Get more detailed responses | Toggle **Advanced** on. |
| Format responses for documentation | Toggle **Doc** on. |
| Change the assistant's persona | Use the **None** dropdown. |
## Exploring Your Data with the Graph View
The Graph view helps you visualize how different pieces of information in your dataset connect to each other. Think of it as a map showing relationships between concepts.
### What You'll See in the Graph
* **Blue dots (nodes):** Each dot represents a piece of information or concept.
Each node can represent a document, concept, extracted entity, or data point.
* **Connecting lines (edges):** Lines connecting dots show how concepts relate to each other.
Edges indicate semantic similarity, direct references, shared attributes, or contextual connections.
* **Clusters:** A cluster is a group of closely connected nodes that represent related topics.
* **Peripheral nodes**: Peripheral nodes are ideas that sit on the edges of the network with fewer relationships.
### Interpreting Your Graph
The information bar displays key metrics:
* **Nodes:** Total number of entities (e.g., "*Nodes: 16*")
* **Edges:** Total number of connections (e.g., "*Edges: 61*")
* **Matches**: Number of nodes that meet the current filter or selection criteria (e.g., "Matches: 34")
* **Zoom:** Current zoom level (e.g., "*86%*")
### **Graph View Controls**
* Pan tool
* Zoom in/out controls
* Fit to screen
* Reset view
* Pause/play simulation
* Globe View toggle (2D/3D)
### Node Search
Search functionality helps you quickly locate specific nodes within your graph:
* **Search by node label**: Find nodes by their display name (e.g., "CustomerFeedback")
* **Search by content keywords**: Locate nodes containing specific keywords within their content
* **Search by unique node ID**: Navigate directly to a node using its identifier (e.g., "65b1143b-8...")
* **Matching results highlight**: Search results are highlighted in the graph for easy identification
* **Navigate to results**: Press Enter or click a search result to navigate directly to that node
#### Node Search Filters
**Options Filter**
* **Quick Jump toggle**: When enabled, automatically focuses on the first matching node in real-time as you type your search query
**Groups Filter**
Filter search results by node groups to narrow down your search scope.
### Mini-Map
Located in the bottom right corner, the mini-map provides an overview of your entire graph structure. A blue square indicates your current viewport position within the larger graph. Click anywhere on the mini-map to quickly navigate to that area. The mini-map is collapsible - use the close button (×) to dismiss it when not needed. This feature is essential for navigating large, complex graphs efficiently.
## Workspace Model Manager
The Workspace Model Manager is the sidebar panel on the right of the Control Center. It groups everything you need to build, configure, and manage a model alongside its dataset and persona. The panel is organized into three tabs — **Model Info**, **Training Data**, and **Recent** — plus a persistent **Quick Actions** panel below them.
The Workspace Model Manager is available on every tab in the platform. It stays docked alongside whatever you're viewing, so you can switch tabs, manage models, and trigger Quick Actions without losing context. If you don't need it visible, toggle the panel closed at any time and reopen it when needed.
### Model Info tab
The default view. Displays core details about the currently active model:
| Field | Description |
| -------------- | ------------------------------------------------------- |
| **Model Name** | The name assigned to this model. |
| **Status** | Current state (e.g., **Ready**, **Building Graph**). |
| **Base Model** | The underlying foundation model (e.g., `gpt-oss-120b`). |
| **Created** | Date the model was created. |
A status of **Ready** means the model is fully configured and available for use.
**Direct base-model selection has been deprecated.** Models deployed directly on the 4MINDS platform now run on `gpt-oss-120b`. To use other foundation models (Claude, Gemini, Llama, Mistral, etc.), connect them through an external provider integration like [Amazon Bedrock](/bedrock), Google Vertex AI, [Amazon SageMaker](/integrations#amazon-sagemaker), or [Microsoft Foundry](/microsoft-foundry).
### Training Data tab
Lists every document currently loaded into the model's dataset. These files form the knowledge base the model draws on when responding to queries. Supported formats include `.txt`, `.pdf`, `.png`, and others — see [Manage your Datasets](/datasets) for the full list.
To add new files, use **Add Data** in the **Current Dataset** card of the Quick Actions panel.
### Recent tab
Shows the most recently accessed models in your workspace. Each entry displays the model name, a short description, and its current status.
The active model is marked with an **OPEN** badge. Click any other entry to switch to it.
## Quick Actions panel
The Quick Actions panel is persistent across all three tabs and provides shortcuts for configuration and management tasks. It's divided into four sections: **Configuration**, **Model**, **Create**, and **Danger Zone**.
### Configuration
* **Current Persona** — Assigns a persona to shape the model's tone and communication style. Defaults to **None**. Use the dropdown to select from available personas. See [Personas](/persona-configuration).
* **Current Dataset** — Shows the dataset currently linked to the model. Click **Add Data** to upload additional files and expand the model's knowledge base.
### Model
| Action | Description |
| ----------------- | ------------------------------------------------------------ |
| **Model Details** | Opens the full details view for the current model. |
| **Edit Model** | Opens the model editor to modify configuration and settings. |
### Create
Build new resources from within the workspace:
| Action | Description |
| --------------------- | --------------------------------------------------------------------- |
| **Create Model** | Creates a new AI model. Expand the option for additional sub-options. |
| **Create Dataset** | Creates a new dataset to attach to a model. |
| **Create Evaluation** | Sets up an evaluation to test model performance. |
| **Create Persona** | Defines a new persona for use in model configuration. |
### Danger Zone
Actions in this section are destructive and may be irreversible. Back up important data before proceeding.
| Action | Description |
| ------------------- | --------------------------------------------------------------------------------- |
| **Reset Workspace** | Resets the entire workspace to its default state. All configurations may be lost. |
| **Delete** | Permanently deletes the current model. This action cannot be undone. |
## Quick reference
| I want to… | Where to go |
| ----------------------------- | ------------------------------------ |
| Check model status | Model Info tab |
| See what data the model uses | Training Data tab |
| Switch to a different model | Recent tab |
| Add more training files | Quick Actions → Add Data |
| Change the model's persona | Quick Actions → Current Persona |
| Create a new model or dataset | Quick Actions → Create section |
| Edit the current model | Quick Actions → Edit Model |
| Delete the model | Quick Actions → Danger Zone → Delete |
# Databricks
Source: https://docs.4minds.ai/databricks
## Overview
The 4MINDS platform integrates with **Databricks**, allowing you to securely connect your Databricks workspace, browse your Unity Catalog, import tables and volume files into 4MINDS datasets, and register Databricks-hosted models as chat models inside 4MINDS. You can also submit Spark and GPU jobs, track ML experiments with MLflow, and share data across organizations via Delta Sharing.
## Diagrams
![Databricks Integration Architecture]()
![Databricks Data Flow Diagrams]()
***
## Getting Started
### Prerequisites
* A Databricks workspace with Unity Catalog enabled
* A SQL warehouse (classic or serverless) that you can query
* One of the following authentication credentials:
* An **OAuth application** configured in your Databricks workspace (recommended), or
* A **Personal Access Token (PAT)** from Databricks, or
* A **Service Principal** (Client ID, Client Secret, Account ID) for automated workloads
* Appropriate Unity Catalog permissions for the catalogs, schemas, tables, and volumes you want to access
### Connecting Your Databricks Workspace
The Databricks integration supports three authentication methods: **OAuth U2M** (recommended), **Personal Access Token**, and **OAuth M2M / Service Principal**.
#### Option A: OAuth U2M (User-to-Machine) — Recommended
1. Open **Integrations** from the main navigation bar in 4MINDS.
2. Find the **Databricks** integration and click **Connect**.
3. Select the **OAuth** tab.
4. Enter your **Workspace URL** (e.g. `https://adb-xxx.azuredatabricks.net`) and **SQL Warehouse ID**.
5. Click **Connect with Databricks**. A popup window will open.
6. Log in to your Databricks account and authorize 4MINDS when prompted.
7. The popup will close automatically once authorization is complete.
> **Note:** OAuth uses OAuth 2.0 with PKCE (Proof Key for Code Exchange). Your Databricks password is never stored by 4MINDS. Access tokens are encrypted and refreshed automatically.
> **Admin setup:** Before users can use OAuth, an admin must configure the Databricks OAuth application (Client ID + Client Secret) once in 4MINDS. See [Admin Setup for OAuth](#admin-setup-for-oauth) below.
#### Option B: Personal Access Token (PAT)
1. Open **Integrations** from the main navigation bar in 4MINDS.
2. Find the **Databricks** integration and click **Connect**.
3. Select the **Personal Access Token** tab.
4. Enter your **Workspace URL**, **Personal Access Token**, and **SQL Warehouse ID**.
* To generate a PAT: log in to Databricks, click your user icon > **User Settings** > **Developer** > **Access tokens** > **Generate new token**.
5. Click **Test Connection** to verify your credentials.
6. Click **Save Credentials** to complete the setup.
> **Note:** Personal Access Tokens are long-lived and do not auto-refresh. If a token is revoked in Databricks, you will need to reconnect with a new token.
#### Option C: Service Principal (OAuth M2M)
For automated pipelines or shared service accounts without a human user:
1. Open **Integrations** from the main navigation bar in 4MINDS.
2. Find the **Databricks** integration and click **Connect**.
3. Select the **Service Principal** tab.
4. Enter your **Workspace URL**.
5. The connection uses the service principal credentials configured by your admin. See [Admin Setup for Service Principal](#admin-setup-for-service-principal) below.
### Disconnecting
To disconnect your Databricks workspace, open **Integrations** from the main navigation bar, find Databricks, and click **Disconnect**. This removes your stored credentials and revokes active tokens.
### Admin Setup for OAuth
Organization admins configure a Databricks OAuth application once per organization:
1. In Databricks, create an OAuth app and note the **Client ID** and **Client Secret**.
2. In 4MINDS, open **Integrations** > **Databricks** > **Admin Settings**.
3. Enter the Client ID and Client Secret, then save.
4. Users in the organization can now connect via the OAuth tab.
The Client Secret is AES-encrypted at rest. Admins can update the Client ID at any time without re-entering the secret; the existing secret is preserved unless a new one is provided.
### Admin Setup for Service Principal
1. In Databricks, create a service principal and generate OAuth credentials (Client ID + Client Secret). Note your **Account ID** from the Databricks account console.
2. In 4MINDS, open **Integrations** > **Databricks** > **Admin Settings**.
3. Enter the service principal's **Client ID**, **Client Secret**, and **Account ID**, then save.
4. Users can now connect via the Service Principal tab.
OAuth U2M and M2M credentials can coexist — users choose which flow to use when connecting.
***
## Authentication Methods
### Method 1: OAuth U2M (User-to-Machine) — Recommended
This is the primary and recommended authentication method. It uses the **OAuth 2.0 Authorization Code grant with PKCE** (Proof Key for Code Exchange), providing the strongest security model for interactive users.
**User connection flow:**
1. The user clicks "Connect with Databricks" and enters their workspace URL and SQL warehouse ID
2. A browser popup opens to the Databricks authorization page
3. The user logs in with their Databricks credentials and grants consent
4. Databricks redirects back to 4MINDS with an authorization code
5. The 4MINDS backend exchanges the code for access and refresh tokens, validating the PKCE code verifier to prevent interception
6. The user's identity (email, name) is retrieved from the Databricks OIDC userinfo endpoint
7. The connection is established — the user sees their email and connection status in the 4MINDS UI
**Scopes requested:** `all-apis`, `offline_access`
**PKCE details:** The platform generates a cryptographically random 32-byte code verifier, computes a SHA-256 code challenge, and sends the challenge with the authorization request. The verifier is submitted during the token exchange, ensuring that even if the authorization code is intercepted, it cannot be used without the original verifier.
**Token lifecycle:**
* Access tokens are refreshed automatically when they are within 5 minutes of expiration. Users never need to re-authenticate unless the refresh token is revoked.
* If Databricks returns a new refresh token during renewal, the updated token is stored immediately (refresh token rotation support).
* Tokens are stored server-side only — they are never sent to or exposed in the browser.
* A CSRF state token is validated on every OAuth callback to prevent cross-site request forgery.
### Method 2: Personal Access Token (PAT)
For users or environments where OAuth is not configured, the integration supports connecting with a Databricks Personal Access Token. This is the simplest method and is useful for quick setup, testing, or workspaces that haven't configured an OAuth application.
**How it works:**
* The PAT is sent as a `Bearer` token in the `Authorization` header on all Databricks API requests
* PATs do not expire automatically but can be revoked by the user in their Databricks workspace settings
* No automatic token refresh is needed since PATs are long-lived
* The token is encrypted at rest and passed to the backend Databricks client for each API call
**Trade-offs vs OAuth:**
* Simpler to set up (no admin OAuth app configuration needed)
* Less secure (static token vs short-lived rotating tokens)
* No user identity verification (the platform trusts whoever provides the token)
* No refresh mechanism (if the PAT is revoked, the user must manually reconnect)
### Method 3: OAuth M2M (Machine-to-Machine) — Service Principal
For automated workflows and service accounts, the integration supports **OAuth 2.0 Client Credentials** grant using a Databricks service principal. This is designed for scenarios where no interactive user is present.
**How it works:**
1. The user selects "Connect with Service Principal" and enters their workspace URL
2. The backend retrieves the M2M credentials from the admin configuration
3. A token is requested directly from the Databricks accounts-level OIDC endpoint (`https://accounts.cloud.databricks.com/oidc/accounts/{account_id}/v1/token`) using the `client_credentials` grant
4. The access token is stored and the connection is established
**Scope requested:** `sql`
**Token lifecycle:**
* M2M tokens are not refreshed — when one expires, a new token is obtained using the same client credentials
* No refresh token is issued (standard behavior for client credentials grants)
* The connection shows as "Service Principal" in the UI rather than a user email
**When to use M2M:**
* Scheduled or automated data pipelines that run without user interaction
* Shared service accounts where individual user OAuth is not practical
* Environments with service principal-based access controls in Unity Catalog
### Authentication Summary
| | OAuth U2M | Personal Access Token | OAuth M2M |
| -------------------- | ------------------------------------------------ | ---------------------------------- | --------------------------------------------------------- |
| **Security** | Highest (PKCE, short-lived tokens, auto-refresh) | Moderate (static long-lived token) | High (OAuth, but shared identity) |
| **User interaction** | One-time popup authorization | Enter token manually | None (admin configures credentials) |
| **Token refresh** | Automatic | Not applicable | New token on expiry |
| **Identity** | User email from Databricks | None (anonymous) | Service principal |
| **Admin setup** | Configure OAuth app (Client ID/Secret) | None | Configure service principal (Client ID/Secret/Account ID) |
| **Best for** | Interactive users, production environments | Quick setup, testing, development | Automated pipelines, service accounts |
### Multi-Tenant Credential Management
Each organization in 4MINDS manages their own Databricks OAuth credentials independently:
* **OAuth U2M:** Admins configure their workspace's OAuth Client ID and Client Secret through the 4MINDS UI. Secrets are encrypted at rest with AES. Admins can update credentials without disconnecting existing users.
* **OAuth M2M:** Admins configure their service principal's Client ID, Client Secret, and Account ID separately. These can coexist alongside U2M credentials — an organization can have both configured simultaneously.
* **Environment variable fallback:** For simpler or single-tenant deployments, OAuth credentials can also be set via environment variables, which serve as a fallback when no per-organization configuration exists.
***
## Unity Catalog Integration
### Data Discovery
The integration provides full hierarchical browsing of Unity Catalog, matching the structure users see in Databricks:
```
Workspace
└── Catalogs
└── Schemas
├── Tables (MANAGED, EXTERNAL, VIEW)
└── Volumes
└── Files & Folders
```
Users navigate this tree in the 4MINDS UI with breadcrumb navigation. At each level, metadata is displayed including owner, description, creation date, and data source format.
### What We Access
| Object | Description |
| -------------- | ---------------------------------------------------------------------------------------------------------------------- |
| **Catalogs** | All accessible catalogs in the workspace |
| **Schemas** | Schemas within a selected catalog |
| **Tables** | Tables within a schema, including type (managed, external, view, Iceberg) and column metadata (names, types, comments) |
| **Volumes** | Unity Catalog volumes and their recursive folder/file contents |
| **Table data** | Preview and export via SQL queries executed through a SQL warehouse |
### SQL Warehouse Handling
The integration detects whether a customer's SQL warehouse is serverless or classic, and adapts accordingly:
* Stopped warehouses are automatically started before queries
* Serverless warehouses get shorter polling intervals (they start faster)
* Warehouse type and serverless status are surfaced in the UI so users know what they're running on
***
## Importing Data
### Creating a Dataset with Databricks Data
1. Create a new dataset (or edit an existing one).
2. Select **Databricks** as a data source.
3. The platform checks your Databricks connection. If not connected, you will be prompted to connect first.
4. Browse your Unity Catalog — select a catalog, then a schema, then pick tables or volume files.
5. Preview the selection before committing to a full import.
6. Click **Add** to stage the selected tables/files for import.
7. Complete the dataset creation to trigger the import.
### How Table Imports Work
Table imports use the **Databricks SQL Statement Execution API**. The platform runs a `SELECT` query through your configured SQL warehouse, exports results as JSON or CSV (selectable by the user), uploads the results to Azure Blob Storage, and processes them through the 4MINDS ETL pipeline.
* You can limit rows, filter columns, and preview the data before importing.
* Stopped warehouses are auto-started before the query runs.
### How Volume File Imports Work
Volume files are downloaded directly from Unity Catalog volumes via the Databricks Files API. Files are transferred to Azure Blob Storage and processed by the same 4MINDS ETL pipeline used for all dataset sources. Supported file types include JSON, CSV, Parquet, XLSX, and text/document formats.
### Combining with Other Sources
Databricks data can be combined with files from other sources in the same dataset. For example, you can import a Unity Catalog table alongside Google Drive documents, Gong transcripts, or uploaded files.
***
## Dataset Sync
### Overview
Beyond one-time imports, 4MINDS supports **automatic, continuous synchronization** with Databricks Unity Catalog volumes. When you enable dataset sync on an imported dataset, 4MINDS monitors the source volume for new and modified files and automatically pulls them in — keeping your dataset up to date without manual re-imports.
This works similarly to `rsync`: the platform maintains a manifest of every file it has already synced (tracking file path, size, and modification timestamp), and on each sync cycle only downloads files that are new or have changed.
### How to Set Up Sync
1. Import files from a Databricks Unity Catalog volume into a 4MINDS dataset (using the standard import flow above).
2. On the dataset, toggle **Dataset Sync** on.
3. Select a sync frequency (see table below).
4. From that point on, 4MINDS automatically checks the source volume at the configured interval and pulls in new or modified files.
### Sync Frequencies
| Frequency | Interval | Best For |
| ---------------- | -------- | ------------------------------------------------------------- |
| **Every minute** | 1 minute | Real-time dashboards, rapidly changing data (Enterprise tier) |
| **Hourly** | 1 hour | Frequently updated data pipelines (Teams & Enterprise) |
| **Daily** | 24 hours | Standard business reporting (All paid tiers) |
| **Weekly** | 7 days | Slowly changing reference data |
| **Monthly** | 30 days | Compliance snapshots, archival data |
### How It Works Internally
**Event-driven scheduler:** The sync system uses an intelligent, event-driven scheduler rather than polling. It calculates the exact time each sync is due based on the configured frequency and last sync timestamp, then sleeps until the earliest next sync. For example, a daily sync that last ran at 8:00 AM will sleep exactly 24 hours — not poll every 30 minutes. When a user creates or modifies a sync configuration, the scheduler wakes up immediately to accommodate the change.
**Change detection:** On each sync cycle, the platform fetches the full file listing from the configured Databricks volume path (recursively including all subfolders). It compares each file against its internal manifest using the file's unique path, size, and modification timestamp. Files are classified as:
* **New** — File path not in the manifest (never seen before)
* **Modified** — File path exists in the manifest but size or modification time has changed
* **Unchanged** — File matches the manifest exactly (skipped)
Only new and modified files are downloaded, which minimizes bandwidth and processing time.
**File processing pipeline:** Changed files are downloaded from the Databricks volume via the Files API, uploaded to Azure Blob Storage (scoped to the user's dataset), and processed through the 4MINDS ETL pipeline. The manifest is updated with the new file metadata after successful processing. Sync statistics (total files synced, total size, last sync status) are tracked and visible to the user.
**Concurrent processing:** Multiple dataset syncs can run in parallel. The scheduler processes up to 5 sync configurations concurrently in each batch, with each sync getting its own isolated database session to prevent conflicts.
### Authentication for Automated Sync
Since dataset sync runs in the background without user interaction, the platform handles authentication automatically:
1. **M2M (Service Principal) — preferred for sync:** If the organization has configured M2M OAuth credentials, the sync system uses the service principal to obtain a fresh access token for each sync cycle. This is the ideal approach for automated workloads because it requires no user interaction and the token is always fresh.
2. **U2M (User OAuth) — fallback:** If M2M is not configured, the sync system uses the user's existing OAuth connection. It automatically refreshes the access token using the stored refresh token when needed, and persists the updated tokens back to the database so subsequent syncs continue to work.
3. **Personal Access Token — simplest:** If the user connected with a PAT, the sync system uses it directly. Since PATs are long-lived, no refresh is needed unless the user revokes the token.
### Dormant Mode
When no sync configurations exist across the entire platform, the scheduler enters dormant mode — checking only once every 5 minutes for newly created configurations. This ensures zero overhead when the feature is not in use. As soon as a sync configuration is created, the scheduler exits dormant mode and resumes event-driven scheduling.
***
## Model Serving Endpoints
### Overview
4MINDS integrates with **Mosaic AI Model Serving** so that models hosted in your Databricks workspace can be used directly inside 4MINDS as chat models. You can discover all serving endpoints you have access to, review their readiness and classification, and register any endpoint as an external model — making it available in the 4MINDS model picker alongside first-party providers.
### Discovering Endpoints
1. In the 4MINDS model picker, open the **Databricks Models** section.
2. 4MINDS calls `GET /databricks/serving-endpoints` and lists every serving endpoint your connected identity can see.
3. Each endpoint shows its name, state (READY / NOT\_READY), classification, and relevant metadata.
For each endpoint, the integration returns:
* **Name, state, creator, and creation timestamp**
* **Served entities** — the raw list of entities backing the endpoint
* **Task** — e.g. `llm/v1/chat`, `llm/v1/completions`
* **Endpoint type** — e.g. `STANDARD`, `FOUNDATION_MODEL_API`
* **Entity type** — `FOUNDATION_MODEL`, `PT_FOUNDATION_MODEL`, `UC_MODEL`, or `EXTERNAL_MODEL`
* **External model provider/name** — e.g. `openai` / `gpt-4o`, `anthropic` / `claude-3-opus`, `custom`
* **`model_type`** — a derived classification 4MINDS applies to each endpoint (see below)
### Model Type Classification
4MINDS classifies every discovered endpoint into one of the following categories so the UI can present them cleanly:
| `model_type` | Description |
| ---------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| `DATABRICKS_FM_PPT` | Databricks-hosted foundation model, pay-per-token (name begins with `databricks-`, entity is `FOUNDATION_MODEL`) |
| `DATABRICKS_FM_PT` | Provisioned-throughput foundation model (`PT_FOUNDATION_MODEL`) |
| `DATABRICKS_FM_UC_SYSTEM_AI` | UC model under `system.ai.*` |
| `DATABRICKS_FM_UC_AGENTS` | UC model with an `llm/v1/*` task — treated as an agentic/chat model |
| `DATABRICKS_CLASSIC_ML` | UC model with no task — classic ML model |
| `FM_EXTERNAL_MODEL` | External provider proxied through Databricks (OpenAI, Anthropic, etc.) |
| `FM_EXTERNAL_MODEL_CUSTOM` | External model with a custom provider |
| `AGENT_BRICKS_KA` | Agent Bricks — Knowledge Assistant |
| `AGENT_BRICKS_MAS` | Agent Bricks — Multi-Agent Supervisor |
| `AGENT_BRICKS_KIE` | Agent Bricks — Knowledge / Information Extraction |
| `AGENT_BRICKS_MS` | Agent Bricks — Model Specialization |
Agent Bricks endpoints are detected via `tile_endpoint_metadata.problem_type`.
### Registering an Endpoint as a 4MINDS Model
1. In the Databricks Models section of the model picker, select an endpoint and click **Register**.
2. Enter a **display name** and optional description.
3. Configure optional model metadata: `max_tokens` (default 4096), `supports_streaming` (default true), `context_window`, `parameters` (e.g. `"8B"`, `"70B"`), `inference_speed`, and a linked `dataset_id`.
4. Save. The endpoint now appears in your model picker alongside first-party providers.
Under the hood, this calls `POST /databricks/register-model` and persists the endpoint as an external model in 4MINDS.
### Chatting with a Registered Endpoint
Once registered, select the model in any 4MINDS chat. Requests are proxied through the same Databricks connection that discovered it, using your stored OAuth/PAT credentials. Responses stream back to the UI just like first-party models.
***
## Serverless GPU Compute
### What We Support
The integration supports submitting deep learning and ML workloads to Databricks Serverless GPU Compute through the Jobs API. Users can run GPU-accelerated Python scripts for model training, fine-tuning, inference, and other custom AI workloads.
### Supported Accelerators
| GPU | Best For | Multi-GPU | Multi-Node |
| --------------- | ---------------------------------------------------------------------- | ---------------- | ---------------- |
| **NVIDIA A10** | Fine-tuning smaller models, classic ML, computer vision, inference | Yes | Yes |
| **NVIDIA H100** | LLM fine-tuning, large-scale model training, distributed deep learning | Up to 8 per node | No (single node) |
A10 is the default GPU when none is specified. The `num_gpus` parameter (per node) is validated at submission — H100 jobs are capped at 8 GPUs because H100 is single-node.
### How It Works
When a GPU job is submitted through 4MINDS:
1. The platform automatically configures the job for serverless GPU compute (GPU jobs always run serverless)
2. Required dependencies (`serverless_gpu`, `torch`) are auto-injected into the job environment
3. The GPU environment version is selected (separate from the CPU serverless environment)
4. The job is submitted via the Databricks Jobs API with the specified GPU type and count
5. The user's Python script uses the `serverless_gpu` library's `@distributed` decorator to leverage GPU resources
### Managed Environments
Two base environments are available for GPU workloads:
* **Default** — Minimal environment with stable client APIs. Best for users who want full control over their dependencies.
* **AI** — Pre-installed with PyTorch, Transformers, Ray, XGBoost, and other popular ML libraries. Best for getting started quickly with training workloads.
### Limitations
* H100 accelerators are single-node only (up to 8 GPUs in one node)
* Only Python workloads are supported
* Additional Databricks-side limits (maximum workload runtime, Private Link support, regional availability) apply as documented by Databricks
***
## Additional Capabilities
### Delta Sharing
The integration supports Databricks Delta Sharing for secure cross-organization data access:
* Create and manage Delta Shares
* Add and remove tables from shares
* Create sharing recipients (token-based or Databricks-to-Databricks)
* Manage share permissions (grant/revoke SELECT access)
### MLflow Integration
Users can create MLflow experiments and log metrics from within 4MINDS:
* Create experiments with custom artifact storage locations and tags
* Log metrics to MLflow runs with step and timestamp tracking
### Spark Job Submission
Beyond GPU workloads, the integration supports submitting general Spark jobs using:
* **Classic compute** — With user-defined cluster configuration
* **Serverless CPU compute** — Using Spark Connect APIs with pip-based dependency management
***
## User-Agent Telemetry
All HTTP requests from 4MINDS to Databricks APIs include the following User-Agent header:
```
User-Agent: 4MINDSPlatform
```
This header is sent consistently on **every** request to Databricks, across all API surfaces:
* **OAuth operations** — Token exchange, token refresh, user info retrieval, M2M token acquisition
* **Unity Catalog** — Catalog, schema, table, and volume listing; table metadata retrieval
* **SQL execution** — Statement execution via SQL warehouses
* **Warehouse management** — Warehouse info queries and auto-start operations
* **Jobs** — Spark and GPU job submission
* **MLflow** — Experiment creation and metrics logging
* **Delta Sharing** — Share and recipient management
* **Model Serving** — Serving endpoint discovery and invocation
* **File operations** — Volume file listing and downloads
The header is set centrally in the Databricks API client, so any new API calls added in the future will automatically include it. There are no code paths that make Databricks API calls without the User-Agent header.
***
## Troubleshooting
| Issue | Solution |
| ---------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **"No Databricks connection found"** | Open Integrations from the main nav and connect your Databricks workspace using OAuth, PAT, or Service Principal. |
| **"Invalid workspace URL"** | Verify your workspace URL is the full HTTPS URL (e.g. `https://adb-xxx.azuredatabricks.net`) and does not include trailing paths. |
| **"Warehouse not found" or "Warehouse ID invalid"** | Copy the warehouse ID from Databricks SQL > Warehouses. The ID is a short alphanumeric string, not the warehouse name. |
| **OAuth popup blocked** | Enable popups for the 4MINDS site in your browser settings, then try again. |
| **OAuth error: "redirect\_uri mismatch"** | Your admin needs to register the 4MINDS callback URL in the Databricks OAuth app configuration. |
| **401 Unauthorized on API calls** | Your credentials may have expired or been revoked. For OAuth, try refreshing the page (token auto-refreshes). For PAT, verify the token is still active in Databricks > User Settings > Access tokens. If issues persist, disconnect and reconnect. |
| **403 Forbidden on a catalog/schema/table** | Unity Catalog permissions. The integration respects Databricks access controls — you can only see objects your identity has `USE CATALOG` / `USE SCHEMA` / `SELECT` permissions for. Contact your Databricks admin. |
| **Warehouse takes a long time to start** | Classic warehouses take 2–5 minutes to start from stopped state; serverless warehouses start in under a minute. The platform polls automatically; larger warehouses may need longer. |
| **Table import fails with "query timeout"** | The SQL warehouse may be under heavy load or the table is very large. Try filtering rows or selecting specific columns, or use a larger warehouse. |
| **Volume file import missing some files** | Verify you have `READ FILES` permission on the volume. Files you cannot see will be silently skipped. |
| **Sync is not pulling new files** | Check the sync status on the dataset. Verify the source volume still exists and your credentials are still valid. Sync uses the customer's OAuth/PAT/M2M credentials — if they are revoked, sync will fail. |
| **Service Principal connection fails** | Verify the admin has configured Client ID, Client Secret, and **Account ID** correctly. The Account ID is required for the accounts-level token endpoint. |
| **Model serving endpoint shows NOT\_READY** | The endpoint is still deploying or has encountered an error in Databricks. Check the endpoint status directly in Databricks > Serving. |
| **GPU job submission fails with "GPU type invalid"** | Only `A10` and `H100` are supported. H100 jobs are capped at 8 GPUs per node and single-node only. |
***
## Security & Privacy
### Authentication
* **OAuth 2.0 with PKCE** — SHA-256 code challenge prevents authorization code interception
* **CSRF protection** — Random state tokens validated on every OAuth callback
* **Encrypted secrets** — All client secrets, PATs, and tokens stored with AES encryption at rest
* **Minimal scopes** — Only the scopes needed for the integration are requested (`all-apis`, `offline_access` for U2M; `sql` for M2M)
* **Automatic token refresh** — Users stay authenticated without manual intervention; refresh token rotation is supported
* **Server-side token storage** — OAuth tokens are never sent to or stored in the browser
### Data Access
* **Respects Unity Catalog permissions** — Users can only access data they have permissions for in Databricks. The integration does not elevate or bypass any Databricks access controls.
* **No persistent data caching** — Table data is queried fresh on each request; no local copies are retained in 4MINDS outside of the imported dataset
* **Scoped cloud storage** — Imported data is uploaded to Azure Blob Storage with per-user/per-dataset scoped access
* **Credential isolation** — Each user's connection credentials are stored independently; no shared tokens across users
* **Per-organization OAuth apps** — Each organization configures its own Databricks OAuth application; credentials are never shared across organizations
***
## FAQ
**Q: What Databricks data can I access?**
A: Anything your identity has Unity Catalog permissions for — catalogs, schemas, tables (managed, external, views, Iceberg), and volumes. Table data is queried through your SQL warehouse; volume files are downloaded via the Files API.
**Q: Do I need Unity Catalog to use this integration?**
A: Yes. The integration is built around Unity Catalog for data discovery and access control.
**Q: Can I import data from an existing dataset?**
A: Yes. You can add Databricks tables or volume files to both new and existing datasets, and combine them with data from other sources.
**Q: How often does dataset sync run?**
A: You choose the frequency when setting up sync: every minute, hourly, daily, weekly, or monthly. Frequencies below daily require a paid tier.
**Q: What happens to my data if I disconnect my Databricks workspace?**
A: Previously imported data remains in your datasets. Automatic syncing will stop, and you will not be able to import new data from Databricks until you reconnect.
**Q: Can I choose which tables or files to sync?**
A: Yes. Sync is configured per-dataset and operates on the volume path you originally imported from. You can enable/disable sync and change frequency at any time.
**Q: Are OAuth tokens exposed to my frontend?**
A: No. OAuth tokens are stored server-side only. The browser only ever sees the connection status.
**Q: Can I connect multiple Databricks workspaces?**
A: Each user account in 4MINDS supports one Databricks connection at a time. To switch workspaces, disconnect the current connection and reconnect with different credentials.
**Q: Does 4MINDS respect my Databricks permissions?**
A: Yes. Every API call is made with your OAuth/PAT/service principal credentials. You can only see and query data that Databricks itself allows you to access.
**Q: Can I use my Databricks-hosted models in 4MINDS chats?**
A: Yes. Discover your workspace's Mosaic AI Model Serving endpoints in the model picker, register any endpoint as a 4MINDS model, and it becomes available alongside first-party providers. See [Model Serving Endpoints](#model-serving-endpoints).
**Q: Does 4MINDS support Agent Bricks?**
A: Yes. Agent Bricks endpoints (Knowledge Assistant, Multi-Agent Supervisor, Knowledge/Information Extraction, Model Specialization) are detected and classified automatically. They can be registered and used as chat models.
**Q: What GPU types are supported for Spark jobs?**
A: NVIDIA A10 (multi-GPU, multi-node) and NVIDIA H100 (up to 8 GPUs, single-node). A10 is the default if none is specified.
**Q: Is there a User-Agent identifying 4MINDS on all requests?**
A: Yes. Every request to Databricks includes `User-Agent: 4MINDSPlatform`. See [User-Agent Telemetry](#user-agent-telemetry).
# Manage your Datasets
Source: https://docs.4minds.ai/datasets
A dataset is a collection of documents or data used to personalize your model. Datasets in the 4MINDS platform can include PDFs, text files, spreadsheets, and other formats. When you upload a dataset, the platform automatically processes it through the ETL and Graph engines to build intelligent knowledge structures. The Datasets tab lets you search, filter, and manage all your training data in one place. Filter datasets by status or data type to find what you need. Create new datasets to start training custom models.
![Screen Shot2025 10 09at2 57 50PM Pn]()
![Screen Shot2025 10 09at2 57 50PM Pn]()
## Dataset and model relationships
Understanding how datasets connect to models is important when planning your data architecture:
* **One model is connected to one dataset.** A model cannot be connected to multiple datasets simultaneously, including at inference time. If you need a model to work with different data, train a separate model on that dataset.
* **One dataset can be connected to multiple models.** You can reuse the same dataset across multiple models — for example, to train variants with different base models or personas.
## Supported File Types
The 4MINDS platform accepts the following file formats for dataset uploads:
* **Text files** (.txt) - Plain text documents
* **Markdown files** (.md) - Formatted text documents with markup
* **CSV files** (.csv) - Comma-separated value spreadsheets
* **JSON files** (.json) - Structured data in JSON format
* **Parquet files** (.parquet) - Columnar storage format **(not supported for Hugging Face imports)**
* **PDF files** (.pdf) - Portable document format files
* **Word documents** (.docx) - Microsoft Word documents
* **Excel spreadsheets** (.xlsx) - Microsoft Excel workbooks
* **JPEG images** (.jpg, .jpeg) - Compressed image files
* **PNG images** (.png) - Portable network graphics
* **GIF images** (.gif) - Graphics interchange format
* **BMP images** (.bmp) - Bitmap image files
* **TIFF images** (.tiff) - Tagged image file format
* **ZIP archives** - Compressed folders containing multiple files
## Automatic OCR Processing
The 4MINDS platform automatically extracts text from images and scanned documents using built-in Optical Character Recognition (OCR). This feature works seamlessly across all base models, no configuration required.
**When OCR is used:**
* **PDF files** with scanned or non-selectable text
* Image files (**JPG, PNG, TIFF, BMP, GIF**) containing text
* Documents with embedded images
**How it works:** When you upload files, our Reflex Router™ automatically detects content that requires OCR processing and extracts the text. The extracted content is then made available for model training and inference, just like any other text data.
**Key benefits:**
* Works with any base model you select for inline tuning
* No manual configuration needed
* Seamlessly integrated into the data processing pipeline
OCR accuracy depends on image quality and resolution. For best results, use clear, high-resolution scans.
## Upload Size Limit
* You can upload up to 100 MB of data at a time. This applies to single files, multiple files, or integration datasets. A progress bar will display the total upload size.
* To upload more data, simply reopen the dataset and upload the next 100 MB batch.
There is no limit on the overall dataset size, only on each individual upload batch.
## Adding Data to Existing Datasets
As your business evolves, your model's knowledge needs to evolve with it. Adding new data to existing datasets keeps your AI current and effective without starting from scratch.
### Why Continuous Data Updates Matter
**Maintain accuracy** - Product features change, policies update, and new edge cases emerge. Without fresh data, your model provides outdated information that frustrates users and erodes trust.
**Capture new patterns** - Each customer interaction reveals new ways people describe problems, ask questions, or use your product. Adding these examples helps your model understand diverse communication styles.
**Improve coverage** - Initial training datasets rarely cover every scenario. As you discover gaps in your model's knowledge, you can fill them by adding targeted data.
**Adapt to business changes** - New products, services, pricing models, or support processes require corresponding updates to your training data.
### How to Add New Data
#### From the Datasets tab
1. Go to the **Datasets** tab
2. Find the dataset you want to update and click the **⋮** menu on the right
3. Select **Edit**
![Screenshot 2026 06 10 At 6 02 17 PM]( "Screenshot 2026 06 10 At 6 02 17 PM")
![Screenshot 2026 06 10 At 6 02 17 PM]( "Screenshot 2026 06 10 At 6 02 17 PM")
4. In the **Edit Dataset** panel (Step 1 of 3 — Setup), update the name, description, or tags as needed, then click **Add Data**
![Screenshot 2026 06 10 At 6 06 33 PM]( "Screenshot 2026 06 10 At 6 06 33 PM")
![Screenshot 2026 06 10 At 6 06 33 PM]( "Screenshot 2026 06 10 At 6 06 33 PM")
5. On the **Upload Data** step (Step 2 of 3), choose your source — **Upload**, **Integrations**, or **URL** — and add your files
![Screenshot 2026 06 10 At 6 06 47 PM]( "Screenshot 2026 06 10 At 6 06 47 PM")
![Screenshot 2026 06 10 At 6 06 47 PM]( "Screenshot 2026 06 10 At 6 06 47 PM")
6. Click **Next** to proceed to the **Review & Update** step (Step 3 of 3), where you can confirm your changes
![Screenshot 2026 06 10 At 6 07 20 PM]( "Screenshot 2026 06 10 At 6 07 20 PM")
![Screenshot 2026 06 10 At 6 07 20 PM]( "Screenshot 2026 06 10 At 6 07 20 PM")
7. Click **Update Dataset** to save
The platform automatically processes and integrates the new data into your knowledge graph.
#### From the Model tab
1. Navigate to Models tab
2. Select the model you want to update with additional data.
3. Click the three-dot menu (⋮) in the Actions column for the model you want to update with additional data.
4. Click the **Add Training Data** button in the shortcuts section
![Screenshot 2026 06 10 At 6 35 50 PM]( "Screenshot 2026 06 10 At 6 35 50 PM")
![Screenshot 2026 06 10 At 6 35 50 PM]( "Screenshot 2026 06 10 At 6 35 50 PM")
5. Click **Add More Data**
6. Choose your data source: **Upload Files**, **Integrations**, or **URL**
7. Upload your new data based on the selected source
8. The data is processed and added to your model's knowledge graph automatically
#### From the Control Center
1. Open your model in the **Control Center**
2. Navigate to **Training Data** tab in the Quick Action panel
3. Click **Add Data**
![Screenshot 2026 06 10 At 6 42 27 PM]( "Screenshot 2026 06 10 At 6 42 27 PM")
![Screenshot 2026 06 10 At 6 42 27 PM]( "Screenshot 2026 06 10 At 6 42 27 PM")
4. Click **Add More Data**
5. Choose your data source: **Upload Files**, **Integrations**, or **URL**
6. Upload your new data based on the selected source
7. The data is processed and added to your model's knowledge graph automatically
New data is automatically integrated into your existing knowledge graph. Nodes and edges update to reflect the new information without disrupting existing knowledge structures.
### Best practices for Data Updates
**Batch related data together** - When creating a dataset, upload related documents together in the same batch rather than dumping unrelated data all at once. Data uploaded together produces stronger graph connections, resulting in richer knowledge structures and better model performance.
**Add incrementally** - Rather than waiting to upload large batches, add new data regularly as it becomes available. This keeps your model current and makes it easier to track what information was added when.
**Document your updates** - Keep notes on what data you added and why. This helps you understand model behavior changes and plan future updates.
**Test after updates** - Use the **Inference Model** feature to verify that new data is being used correctly and hasn't introduced conflicts with existing knowledge.
**Mix data types** - When adding new information, include multiple formats when possible. For example, if you're adding a new product feature, include documentation (PDF), example support tickets (CSV), and screenshots (images).
**Retrain when needed** - After significant data additions, retrain your model to fully integrate the new knowledge. Minor updates may not require retraining, but substantial changes benefit from it.
## Building Comprehensive Datasets
Training an effective AI model requires more than uploading a single file type. Just as you wouldn't hire a customer support agent and only give them a product manual, your model needs diverse perspectives and contexts to develop true understanding.
### Example: Training on Customer Support Excellence
Let's say you want your model to handle customer support inquiries effectively. Here's how to structure a robust, multimodal dataset using 4MINDS' supported formats:
**Visual understanding (images & screenshots)**
Upload visual content showing real customer interactions:
* **Product interfaces** - Screenshots of your software, dashboard views, error messages, feature locations
* **Troubleshooting visuals** - Common configuration issues, installation steps, system architecture diagrams
* **Documentation** - Annotated screenshots showing workflows, setup guides, integration diagrams
* **Error states** - What customers see when things go wrong, loading states, failure modes
* **Customer-submitted images** - Photos of hardware issues, setup problems, packaging damage
* **Competitor products** - Interface comparisons, feature differences, migration guides
**Conceptual knowledge (PDFs & documents)**
Add comprehensive written content:
* **Product documentation** - Technical specifications, API references, user guides, release notes
* **Internal knowledge bases** - Troubleshooting playbooks, known issues, workaround procedures
* **Policy documents** - SLA agreements, refund policies, terms of service, data privacy guidelines
* **Training materials** - Onboarding docs for new support agents, escalation procedures, quality standards
* **Industry context** - Regulatory compliance guides, security best practices, industry standards
* **Best practices** - Customer service frameworks, communication guidelines, de-escalation techniques
* **Competitive intelligence** - How competitors solve similar problems, market positioning, feature comparisons
**Structured data (CSV & spreadsheet files)**
Include quantitative patterns and history:
* **Support ticket history** - Ticket IDs, timestamps, issue categories, resolution times, customer satisfaction scores
* **Customer data** - Account types, subscription tiers, usage patterns, feature adoption rates
* **Product usage analytics** - Most-used features, error rates, session durations, drop-off points
* **Response metrics** - First response time, resolution time, reopened tickets, escalation rates
* **Customer sentiment** - NPS scores, CSAT ratings, survey responses, sentiment analysis results
* **Seasonal patterns** - Ticket volume by time/day/season, spike events, capacity planning data
* **Agent performance** - Resolution rates, customer satisfaction per agent, specialization areas
**Communication history (email & chat logs)**
Provide real conversation examples:
* **Resolved tickets** - Successful interactions showing problem identification and resolution
* **Escalated cases** - Complex issues requiring multiple touchpoints or specialist involvement
* **Edge cases** - Unusual requests, policy exceptions, creative problem-solving examples
* **Tone variations** - Professional responses, empathetic communications, frustrated customer de-escalation
* **Multi-channel interactions** - Email threads, chat transcripts, phone call summaries, social media responses
* **Follow-ups** - Post-resolution check-ins, proactive outreach, account management communications
**Audio (coming soon)**
Add dynamic training materials:
* **Call recordings** - Customer support calls showing tone, pacing, active listening, problem resolution
* **Product demos** - Video walkthroughs of features, setup processes, advanced use cases
* **Training sessions** - Internal workshops, role-playing scenarios, best practice reviews
* **Customer feedback sessions** - User interviews, usability testing, feature request discussions
**Contextual business data (mixed formats)**
Round out understanding with operational context:
* **Product roadmap** - Upcoming features, deprecation schedules, beta programs
* **Billing systems** - Invoice examples, pricing tiers, renewal processes, refund workflows
* **Integration documentation** - Third-party connections, API partnerships, data sync processes
* **Company information** - Team structure, hours of operation, regional support coverage, contact escalation paths
* **Legal & compliance** - GDPR requirements, data handling procedures, audit trails, security protocols
### Why this Matters
When you combine these diverse data types, your model develops:
* **Contextual problem-solving** that understands not just what the issue is, but why it matters and how it impacts the customer's business
* **Tone awareness** from seeing thousands of interactions, knowing when to be technical vs empathetic, formal vs conversational
* **Pattern recognition** identifying common issues before customers fully describe them, predicting follow-up questions
* **Operational intelligence** understanding SLAs, escalation paths, when to involve specialists, and business constraints
* **Proactive guidance** suggesting solutions based on similar past cases, usage patterns, and product knowledge
A model trained only on product documentation would fail when a frustrated customer describes a problem in non-technical terms, or when an edge case requires policy interpretation. But a model trained with this comprehensive, multimodal approach develops the nuanced intelligence to handle real customer interactions effectively.
## Tutorial: Fine-Tune a Model with Hugging Face Datasets
This tutorial walks you through importing datasets from Hugging Face to train a custom model in 4MINDS.
### What you'll build
By the end of this tutorial, you'll have a custom model trained on Hugging Face data that can:
* Understand domain-specific terminology and concepts
* Extract relevant information from your training data
* Provide accurate, contextual responses to queries in your domain
### Prerequisites
* A 4MINDS account with access to the platform
* A [Hugging Face account](https://huggingface.co/join) with a generated [access token](https://huggingface.co/settings/tokens)
* Basic understanding of the 4MINDS model creation workflow
### Fine-tuning overview
Fine-tuning allows you to customize base models for your specific use case by training them on your own data. The fine-tuning feature enables you to:
* Create custom models tailored to your domain (e.g., financial analysis, customer support)
* Train on proprietary datasets to improve accuracy for specific tasks
* Deploy models via API or test them in the interactive Playground
* Monitor performance metrics including response time, token speed, and success rate
#### Model status types
| Status | Description |
| ------------------ | ------------------------------------------------------------- |
| **Ready** | Model is trained and available for use |
| **Building Graph** | Model is currently being compiled (shows percentage progress) |
| **Training** | Model is actively learning from training data |
| **Archived** | Model is stored but not actively deployed |
#### Base model
Models deployed directly on the 4MINDS platform run on **`gpt-oss-120b`**. In-platform base-model selection has been deprecated.
To use other foundation models (Claude, Gemini, Llama, Mistral, etc.), connect them through an external provider integration like [Amazon Bedrock](/bedrock), Google Vertex AI, [Amazon SageMaker](/integrations#amazon-sagemaker), or [Microsoft Foundry](/microsoft-foundry).
#### Training data best practices
1. **Provide diverse examples** – Include variations of similar questions to improve generalization
2. **Maintain consistency** – Use a consistent format and tone across all training samples
3. **Include edge cases** – Add examples of boundary conditions and unusual queries
4. **Quality over quantity** – 500 high-quality examples often outperform 5,000 poor ones
### Step 1: Access the data upload screen
During the model creation process (Step 3 of 4), you'll reach the **Data Upload** screen. Here you can choose how to provide training data to customize your model.
1. Under **Choose Data Source**, ensure the **Upload New Data** tab is selected
2. You have three options under **Add Files from Sources**:
* **Upload Files** - Local files from your computer
* **Integrations** - External data sources
* **URL** - Import from a web address
3. To import from Hugging Face, click the **Integrations** button
### Step 2: Select Hugging Face integration
On the **Select Integration** screen, you'll see a list of available data source integrations including Amazon S3, Azure Blob Storage, Google Cloud Storage, and others.
1. Scroll down and select **Hugging Face** from the list
If you see "Not configured" next to an integration, you may need to set up credentials first via **Configure Integrations** at the top of the list.
### Step 3: Search for your dataset
The **Import from HuggingFace** screen allows you to search the Hugging Face Hub for datasets.
1. Enter your search query in the search bar (e.g., "finQA")
2. Click **Search**
3. Browse the results using the available tabs:
* **Popular Datasets** – Trending datasets on Hugging Face
* **My Datasets** – Your personal Hugging Face datasets
* **Search Results** – Results matching your query
Each dataset card displays helpful information including:
* Dataset name and author
* Description
* Download count
* Size and format
* Task type and modality
Click on the dataset you want to import.
### Step 4: Configure dataset import settings
On the **Dataset Details** screen, you can configure import settings for your selected dataset.
Review the dataset information:
* Name and author
* Description
* Download statistics
* Task IDs, size, and format
Configure the following options:
* **Configuration** – Select the dataset configuration (e.g., "Default")
* **Split** – Choose which data split to import (e.g., "Test", "Train", "Validation")
When ready, click **+ Add Dataset** to import the files.
### Step 5: Review attached files
After importing, you'll return to the **Data Upload** screen. Your imported files now appear under **Attached Files** with details including:
* File name
* Source (Hugging Face icon)
* File size
* Row count
For example, importing a dataset might result in files like:
* `relevance.jsonl` – 66.68 KB, 341 rows
* `queries.jsonl` – 137.71 KB, 705 rows
* `corpus.jsonl` – 1.44 MB, 7549 rows
**Rsync settings (optional)**
Enable **Rsync Settings** to automatically sync new files from your Hugging Face sources when you log in. This keeps your training data up to date.
Click **Next** to proceed.
### Step 6: Review and launch training
On the **Review & Launch** screen (Step 4 of 4), verify your configuration summary:
| Setting | Value |
| ------------------- | -------------------------- |
| Use Case | Your selected use case |
| Base Model | `gpt-oss-120b` |
| Data Files | Imported from Hugging Face |
| Rsync Configuration | HuggingFace - All folders |
| Persona | Your selection or default |
| Deployment | e.g., Cloud API |
If everything looks correct, click **Confirm & Train** to start the training process.
### Step 7: Monitor training progress
After launching, you'll be taken to the **Models** dashboard in Control Center. Your new model will appear in the list with:
* **Status** – "New" badge with "Building Graph" progress indicator
* **Parameters** – Model size (e.g., 14b)
* **Base** – Base model used (e.g., Phi)
* **Created** – Timestamp
The status will update as training progresses through the pipeline. Once complete, the status will change to **Ready**.
### Step 8: Test in the Playground
The Playground provides an interactive environment to evaluate your fine-tuned model before deployment.
**Accessing the Playground:**
1. From the model dashboard, click the **⋮** menu on any model
2. Select **Run Model**
3. Or navigate to **Control Center → Playground** and select your model
**Playground features:**
* **Real-time responses** – See model outputs as they generate
* **Conversation history** – Maintain context across multiple turns
* **View Graph** – Visualize model reasoning and token flow
* **Clear All Chats** – Reset the conversation history
* **Add Model** – Compare multiple models side-by-side
**Example test queries for a financial analysis model:**
* "What is the ratio of operating income to total revenue?"
* "What is the total of all lease obligations?"
* "What was the percentage change in revenue from 2018 to 2019?"
### Model actions
Access these options via the **⋮** menu on any model:
| Action | Description |
| --------------------- | --------------------------------------------- |
| **Run Model** | Open the model in the Playground for testing |
| **API Access** | View API endpoints and authentication details |
| **Edit Model** | Modify model configuration and settings |
| **Add Training Data** | Upload additional training examples |
| **Full Screen** | Expand the model view |
| **Duplicate** | Create a copy of the model with its settings |
| **Archive** | Move to archived storage (can be restored) |
| **Delete** | Permanently remove the model |
### API integration
Deploy your fine-tuned model via API for production use.
**Getting API credentials:**
1. Click **⋮** on your model
2. Select **API Access**
3. Copy your API endpoint and authentication token
**Example request:**
```bash theme={null}
curl -X POST https://api.4minds.ai/v1/completions \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "customer-faq-expert",
"prompt": "How do I reset my password?",
"max_tokens": 500
}'
```
### Performance optimization
**Improving response quality:**
1. **Add more training data** – Expand coverage of your use case
2. **Refine existing data** – Remove low-quality or contradictory examples
3. **Adjust the persona** – Use "Technical Expert" for specialized domains.
4. **Use an externally connected model when needed** – For complex reasoning beyond `gpt-oss-120b`, connect a larger or specialized foundation model via [Amazon Bedrock](/bedrock), Google Vertex AI, [Amazon SageMaker](/integrations#amazon-sagemaker), or [Microsoft Foundry](/microsoft-foundry).
**Improving speed:**
1. **Optimize prompt length** – Shorter prompts reduce processing time.
2. **Enable caching** – Reuse responses for common queries.
### Troubleshooting
| Issue | Solution |
| ------------------------------- | ---------------------------------------------------------------------------------- |
| Model stuck on "Building Graph" | Large models may take longer; check progress percentage |
| Low success rate | Review training data for errors or inconsistencies |
| Slow response time | Optimize prompts, or route the request through a faster externally connected model |
| Inaccurate responses | Add more diverse training examples |
### FAQs
**Q: How long does training take?**
Training time depends on model size and dataset. Expect 30 minutes to several hours for large models.
**Q: Can I update a model after deployment?**
Yes, use "Add Training Data" to incrementally improve your model.
**Q: What's the difference between Archive and Delete?**
Archived models can be restored; deleted models are permanently removed.
**Q: How many models can I have active?**
Check your plan limits in the account settings.
### Supported file formats
4MINDS supports the following file formats for training data from Hugging Face:
BMP, CSV, DOCX, GIF, HTML, JPEG, JPG, JSON, JSONL, MD, ODT, PARQUET, PDF, PNG, TIFF, TSV, TXT, XLSX
Multiple files are supported per upload.
### Tips
* **Choose appropriate splits** – For fine-tuning, you typically want the "Train" split. Use "Test" or "Validation" for evaluation datasets.
* **Check dataset size** – Larger datasets may take longer to import and process.
* **Enable Rsync** – If you're working with frequently updated datasets, enable Rsync to stay current automatically.
For best results, combine the Hugging Face dataset with your organization's proprietary documents. This creates a model that understands both general concepts and your specific business context.
## Tutorial: Fine-Tune a Model with Hugging Face Datasets via API
This tutorial shows how to fine-tune a 4MINDS model using the FinQA dataset from Hugging Face through the API. Since the API requires manual dataset uploads, you'll download the dataset from Hugging Face and upload it to 4MINDS.
### What you'll build
A custom model trained on financial Q\&A data, created entirely through API calls, ideal for automation and CI/CD pipelines.
### Prerequisites
* A 4MINDS account with API access
* Your API key (found in Account Settings)
* A [Hugging Face account](https://huggingface.co/join) with a generated [access token](https://huggingface.co/settings/tokens)
* Python 3.7+ with the `requests` and `datasets` libraries installed
### Step 1: Download the FinQA dataset from Hugging Face
First, download the FinQA dataset locally using the Hugging Face `datasets` library:
```python theme={null}
from datasets import load_dataset
import json
# Load the FinQA dataset
dataset = load_dataset("ibm/finqa", split="train")
# Convert to JSON format for upload
data = [{"question": item["question"], "answer": item["answer"]} for item in dataset]
# Save to a local file
with open("finqa_training_data.json", "w") as f:
json.dump(data, f, indent=2)
print(f"Saved {len(data)} training examples to finqa_training_data.json")
```
### Step 2: Upload the dataset to 4MINDS
Use the 4MINDS API to create a dataset and upload your file:
```python theme={null}
import requests
API_KEY = "your_api_key_here"
BASE_URL = "https://api.4minds.ai/api/v1"
headers = {
"Authorization": f"Bearer {API_KEY}",
}
# Create a new dataset with the uploaded file
with open("finqa_training_data.json", "rb") as f:
response = requests.post(
f"{BASE_URL}/user/dataset",
headers=headers,
files={"file": ("finqa_training_data.json", f, "application/json")},
data={"name": "FinQA Training Data"}
)
dataset_response = response.json()
dataset_id = dataset_response["id"]
print(f"Created dataset with ID: {dataset_id}")
```
### Step 3: Create a model with the dataset attached
Now create a new model and attach your dataset for training:
```python theme={null}
# Create a new model with the dataset
model_payload = {
"name": "Financial QA Assistant",
"description": "Fine-tuned on FinQA dataset for financial question answering",
"dataset_id": dataset_id
}
response = requests.post(
f"{BASE_URL}/user/model",
headers={**headers, "Content-Type": "application/json"},
json=model_payload
)
model_response = response.json()
model_id = model_response["id"]
print(f"Created model with ID: {model_id}")
print(f"Training status: {model_response['status']}")
```
### Step 4: Monitor training progress
Poll the API to check when training completes:
```python theme={null}
import time
while True:
response = requests.get(
f"{BASE_URL}/user/model/{model_id}",
headers=headers
)
status = response.json()["status"]
print(f"Training status: {status}")
if status == "ready":
print("Training complete!")
break
elif status == "failed":
print("Training failed. Check the dashboard for details.")
break
time.sleep(30) # Check every 30 seconds
```
### Step 5: Test your model via API
Once training completes, send inference requests to your fine-tuned model:
```python theme={null}
# Send a test query
inference_payload = {
"model_id": model_id,
"message": "What was the revenue growth percentage year-over-year?"
}
response = requests.post(
f"{BASE_URL}/user/model/{model_id}/inference",
headers={**headers, "Content-Type": "application/json"},
json=inference_payload
)
print("Model response:")
print(response.json()["response"])
```
### Complete script
Here's the full workflow in a single script:
```python theme={null}
from datasets import load_dataset
import requests
import json
import time
# Configuration
API_KEY = "your_api_key_here"
BASE_URL = "https://api.4minds.ai/api/v1"
headers = {"Authorization": f"Bearer {API_KEY}"}
# Step 1: Download FinQA from Hugging Face
print("Downloading FinQA dataset...")
dataset = load_dataset("ibm/finqa", split="train")
data = [{"question": item["question"], "answer": item["answer"]} for item in dataset]
with open("finqa_training_data.json", "w") as f:
json.dump(data, f, indent=2)
# Step 2: Upload to 4MINDS
print("Uploading dataset to 4MINDS...")
with open("finqa_training_data.json", "rb") as f:
response = requests.post(
f"{BASE_URL}/user/dataset",
headers=headers,
files={"file": ("finqa_training_data.json", f, "application/json")},
data={"name": "FinQA Training Data"}
)
dataset_id = response.json()["id"]
# Step 3: Create model
print("Creating model...")
response = requests.post(
f"{BASE_URL}/user/model",
headers={**headers, "Content-Type": "application/json"},
json={
"name": "Financial QA Assistant",
"description": "Fine-tuned on FinQA for financial Q&A",
"dataset_id": dataset_id
}
)
model_id = response.json()["id"]
# Step 4: Wait for training
print("Waiting for training to complete...")
while True:
response = requests.get(f"{BASE_URL}/user/model/{model_id}", headers=headers)
status = response.json()["status"]
if status == "ready":
break
time.sleep(30)
# Step 5: Test the model
print("Testing model...")
response = requests.post(
f"{BASE_URL}/user/model/{model_id}/inference",
headers={**headers, "Content-Type": "application/json"},
json={"message": "What was the revenue growth percentage?"}
)
print(f"Response: {response.json()['response']}")
```
Store your API key in environment variables rather than hardcoding it. Use `os.environ.get("FOURMINDS_API_KEY")` for production scripts.
# dbt
Source: https://docs.4minds.ai/dbt
Connect 4MINDS to dbt to import model, source, and test metadata for AI-powered analysis
## Overview
The 4MINDS platform integrates with **dbt** (dbt Cloud), letting you import your project's metadata — model definitions, sources, tests, exposures, and account structure — directly into your 4MINDS models. This makes your dbt project's semantic layer available for AI-powered querying, knowledge graph construction, and retrieval.
4MINDS supports two ways to connect:
| Method | Best for | What it can access |
| ----------------- | -------------------------------------------------------------- | --------------------------------------------------------------------------------- |
| **OAuth** | Read-only, no long-lived secret to manage, tokens auto-refresh | Models, sources, tests, exposures, and account structure (projects, environments) |
| **Service Token** | Full read access, including job runs and run artifacts | Everything OAuth can, **plus** job listings and the raw run **manifest** |
> Both methods are read-only against your dbt account. OAuth cannot list jobs or import a run manifest — those operations require a service token.
***
## Getting Started
### Prerequisites
* A dbt Cloud account
* Your account's **Access URL** and numeric **Account ID** (see [Finding your Access URL and Account ID](#finding-your-access-url-and-account-id))
* A 4MINDS account with access to the Integrations page
* For **OAuth**: your dbt account must allow OAuth app registration (App integrations). No manual setup is required — 4MINDS registers itself automatically (see below). App integrations may require an Enterprise-tier dbt plan.
* For **Service Token**: permission to create a service token in dbt
### Finding your Access URL and Account ID
Both connection methods need these two values:
* **Access URL** — your dbt account's base URL. For multi-tenant accounts this is `https://cloud.getdbt.com`. Cell-based or single-tenant accounts use a URL like `https://abc123.us1.dbt.com`. You can find it in dbt under **Account Settings**.
* **Account ID** — the numeric account identifier. It appears in your dbt URL (for example, `https://cloud.getdbt.com/settings/accounts/12345` → the Account ID is `12345`) and in **Account Settings**.
***
## Option 1 — Connect with OAuth
OAuth is the recommended method for read-only metadata access. 4MINDS never stores a long-lived secret, and access tokens are refreshed automatically.
**No manual setup in dbt is required.** 4MINDS registers itself automatically using your account's OAuth app registration (RFC 7591 Dynamic Client Registration), so there's no integration to add or Redirect URI to configure by hand. When you connect, a read-only client named **4MINDS** is created on your dbt account — you'll see it under **Account Settings → Integrations** in the **Dynamically registered** list.
### Connect from 4MINDS
1. Sign in to [4MINDS](https://app.4minds.ai).
2. Navigate to the **Integrations** page from the main navigation bar.
3. Locate the **dbt** integration card and click on it.
4. Choose the **OAuth** tab.
5. Enter your **Access URL** and **Account ID**.
6. Click **Connect with dbt**. A popup opens dbt's authorization screen.
7. Sign in to dbt (or use an existing session) and approve the requested read-only access.
8. Once authorized, the popup closes and the connection is established.
> If the popup is blocked, enable popups for the 4MINDS site in your browser and try again.
> If your dbt account has OAuth app registration disabled, the connect attempt will report that app integrations aren't enabled. Ask a dbt admin to enable App integrations under **Account Settings → Integrations**, or connect with a service token instead.
### OAuth scopes
When you connect via OAuth, 4MINDS requests only read-only scopes:
| Scope | Description |
| ---------------- | ----------------------------------------------------------------------------- |
| `offline_access` | Issues a refresh token so 4MINDS can renew access without prompting you again |
| `account:read` | Read account structure — projects and environments |
| `catalog:read` | Read the project catalog — models, sources, tests, and exposures |
***
## Option 2 — Connect with a Service Token
Use a service token if you need full read access, including listing jobs and importing a run's **manifest**.
### Step 1 — Create a service token in dbt
1. In dbt, go to **Account Settings → Service tokens** (on some accounts this is under **API Access**).
2. Click **Create service token** (or **New token**).
3. Give it a name — for example, `4MINDS`.
4. Assign **permission sets**. 4MINDS reads from two different dbt APIs, so a token needs coverage for both:
| Permission set | Why it's needed | Enables data types |
| ------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------- | -------------------------------------- |
| **Metadata Only** | Read the Discovery / Metadata (GraphQL) API | Models, Sources, Tests, Exposures |
| **Job Admin** (or a read-only account set, such as **Account Viewer** on Enterprise) | Read the Administrative API — projects, environments, jobs, runs, and run artifacts | Metadata (account structure), Manifest |
Assign **both** sets to the same token. Metadata Only alone does not grant Administrative API access, and an admin/job set alone does not grant Discovery access. 4MINDS only ever reads — no write permission is required.
5. Save the token and **copy it immediately** — dbt shows the token value only once.
> The exact permission-set names and their availability depend on your dbt plan. If your plan doesn't expose these granular sets, a broader read-capable set (e.g. Account Admin) also works, but least-privilege (Metadata Only + a read-only job/account set) is recommended.
> Store the token securely. Anyone with the token can read from your dbt account within the granted permission set.
### Step 2 — Connect from 4MINDS
1. Sign in to [4MINDS](https://app.4minds.ai) and open the **Integrations** page.
2. Click the **dbt** integration card.
3. Choose the **Service Token** tab.
4. Enter your **Access URL**, **Account ID**, and the **Service Token** you created.
5. Click **Test Connection** to verify the credentials, then **Save Credentials**.
***
## Disconnecting
To disconnect dbt, open **Integrations** from the main navigation bar, find dbt, and click **Disconnect**. This removes your stored credentials (and, for OAuth, your tokens) from 4MINDS.
***
## Available Data Types
The following dbt data types can be imported into 4MINDS:
| Data Type | Description | OAuth | Service Token |
| ------------- | ------------------------------------------------------------------------- | :---: | :-----------: |
| **Models** | Model definitions — relation, materialization, description, tags, columns | ✅ | ✅ |
| **Sources** | Source definitions — relation, description, freshness | ✅ | ✅ |
| **Tests** | Data test definitions — column, type, last run status | ✅ | ✅ |
| **Exposures** | Exposure definitions — type, owner, maturity, downstream asset | ✅ | ✅ |
| **Metadata** | Account structure — projects, environments, and jobs | ✅ | ✅ |
| **Manifest** | Raw `manifest.json` from the most recent run of a dbt job | — | ✅ |
> Listing jobs and importing a run **manifest** are not available over OAuth (read-only scopes). Connect with a service token to use them.
***
## Importing Data
### Creating a Model with dbt Data
1. Navigate to the **Models** page and click **Create Model**.
2. Provide a **name** and **use case** for the model (e.g., "dbt Catalog" / "Analyze our dbt project metadata"). Press **Next**.
3. On the **Upload Data** step, select the **Integrations** option.
4. Scroll to the **Enterprise** group and choose **dbt**.
5. Select the data types you'd like to import (models, sources, tests, exposures, metadata).
6. Confirm the import and create the model.
Once processing completes, the dataset is available for AI-powered querying, knowledge graph construction, and retrieval on the **Models** page.
### Querying Your Data
1. Find the model you created on the **Models** page.
2. Click the **Run** button.
3. On the chat interface, send a prompt or query to get AI-generated responses about your dbt project.
***
## Troubleshooting
| Issue | Solution |
| ------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **"No dbt connection found"** | Open Integrations from the main nav and connect your dbt account via OAuth or a service token. |
| **OAuth authorization fails / lands on the dbt home screen** | Re-check the Access URL and Account ID, then try connecting again. 4MINDS registers a fresh OAuth client on each attempt, so a simple retry resolves most transient authorization errors. |
| **"Your dbt account hasn't enabled OAuth app integrations"** | Your dbt account blocks OAuth app registration. Ask a dbt admin to enable App integrations under **Account Settings → Integrations**, or connect with a service token instead. |
| **Popup blocked during OAuth** | Enable popups for the 4MINDS site in your browser settings, then try again. |
| **Service token rejected** | Re-check the Access URL, Account ID, and token. Ensure the token's permission set grants read access and that the token hasn't been revoked. |
| **Jobs or manifest import unavailable** | These require a service token — reconnect using the Service Token option. |
| **Connection expired (OAuth)** | 4MINDS refreshes OAuth tokens automatically. If issues persist, disconnect and reconnect. |
***
## Security & Privacy
* **Read-only:** Both connection methods only read from your dbt account.
* **OAuth 2.0 with PKCE:** OAuth connections use industry-standard OAuth 2.0 with PKCE. 4MINDS stores no long-lived secret for OAuth.
* **Encrypted at rest:** Service tokens and OAuth tokens are encrypted at rest.
* **Automatic token refresh:** OAuth tokens are refreshed automatically before they expire.
* **Per-user access:** Each user connects their own dbt account; connections are not shared across users.
***
## FAQ
**Q: Should I use OAuth or a service token?** A: Use OAuth for read-only metadata (models, sources, tests, exposures, account structure) with no secret to manage. Use a service token if you also need to list jobs or import a run manifest.
**Q: Do I need to set up the OAuth integration in dbt first?** A: No. 4MINDS registers itself automatically via Dynamic Client Registration when you connect — there's no integration to add or Redirect URI to configure. The dbt account just needs OAuth app registration (App integrations) enabled. The registered client appears under **Account Settings → Integrations** as **4MINDS** in the **Dynamically registered** list.
**Q: Where do I find my Access URL and Account ID?** A: In dbt under **Account Settings**. The Account ID also appears as the number in your dbt account URL.
**Q: Can I switch from a service token to OAuth (or vice versa)?** A: Yes. Disconnect the current connection, then reconnect using the other method.
**Q: What happens to imported data if I disconnect?** A: Previously imported data remains in your models. You won't be able to import new data from dbt until you reconnect.
# Evaluate your Model Performance
Source: https://docs.4minds.ai/evaluations
Evaluations provide comprehensive model performance reports with key metrics to help you make data-driven decisions about model quality. The 4MINDS platform automatically generates evaluation reports that assess your model's accuracy, response quality, and overall effectiveness. Use Evaluations feature to identify areas for improvement and validate that your model meets production requirements.
To evaluate your model, navigate to the '**Evaluations**' tab and click '**Create Evaluation**'.
![Screen Shot2025 10 31at1 34 59PM Pn]()
![Screen Shot2025 10 31at1 34 59PM Pn]()
Select your evaluation method from the available options:
* **RAGAS Benchmark** (***Currently Available***): A standardized test that measures model performance on specific tasks using predefined metrics and datasets.
* **Model as Judge** (***Currently Available***): Automatically compares your customized model against a base foundation model, with ChatGPT acting as an AI judge to evaluate responses side-by-side and determine which performs better.
* **Model Comparison** (***Coming Soon***)**:** Side-by-side evaluation of multiple models using standardized tests to compare performance, accuracy, and response quality across specific tasks.
* **Human Evaluation** (***Coming Soon***): Manual assessment by human reviewers to evaluate response quality, relevance, and appropriateness based on subjective criteria.
## RAGAS Benchmark evaluation
**Step 1: Choose 'RAGAS Benchmark' evaluation method**
![Screen Shot2025 11 25at2 28 12PM Pn]( "Screen Shot2025 11 25at2 28 12PM Pn")
![Screen Shot2025 11 25at2 28 12PM Pn]( "Screen Shot2025 11 25at2 28 12PM Pn")
Click '**Next**' to continue.
**Step 2: Select your base model**
**Direct base-model selection has been deprecated.** Models deployed directly on the 4MINDS platform run on `gpt-oss-120b`, which is also the default comparison baseline used in evaluations. To compare against other foundation models, connect them through an external provider integration like [Amazon Bedrock](/bedrock), Google Vertex AI, [Amazon SageMaker](/integrations#amazon-sagemaker), or [Microsoft Foundry](/microsoft-foundry).
![Screen Shot2025 11 25at2 37 42PM Pn]( "Screen Shot2025 11 25at2 37 42PM Pn")
![Screen Shot2025 11 25at2 37 42PM Pn]( "Screen Shot2025 11 25at2 37 42PM Pn")
Click '**Next**' to continue.
**Step 3: Review and confirm the details**
![Screenshot 2026 06 10 At 5 07 29 PM]( "Screenshot 2026 06 10 At 5 07 29 PM")
![Screenshot 2026 06 10 At 5 07 29 PM]( "Screenshot 2026 06 10 At 5 07 29 PM")
Review and confirm the details, then click '**Start Evaluation**' to begin.
Monitor the evaluation status in the Evaluation Dashboard. Once complete, the status will update to "**Completed**" and the evaluation report will open in a separate window.
![Screen Shot2025 10 31at1 51 40PM Pn]( "Screenshot2026 02 17at7 55 39PM")
![Screen Shot2025 10 31at1 51 40PM Pn]( "Screenshot2026 02 17at7 55 39PM")
## Model as Judge evaluation
Model as Judge automatically compares your customized model against a base foundation model. ChatGPT evaluates responses side-by-side to determine which performs better.
### Setting up a Model as Judge evaluation
**Step 1: Choose 'Model as Judge' evaluation method**
![Screen Shot2025 11 25at2 26 40PM Pn]( "Screen Shot2025 11 25at2 26 40PM Pn")
![Screen Shot2025 11 25at2 26 40PM Pn]( "Screen Shot2025 11 25at2 26 40PM Pn")
Click '**Next**' to continue.
**Step 2: Select your trained model**
![Screen Shot2025 11 24at3 20 17PM Pn]( "Screen Shot2025 11 24at3 20 17PM Pn")
![Screen Shot2025 11 24at3 20 17PM Pn]( "Screen Shot2025 11 24at3 20 17PM Pn")
Select the customized model you want to evaluate, review its description, and click '**Next**' to continue.
**Step 3: Review and confirm**
![Screen Shot2025 11 24at3 20 49PM Pn]( "Screen Shot2025 11 24at3 20 49PM Pn")
![Screen Shot2025 11 24at3 20 49PM Pn]( "Screen Shot2025 11 24at3 20 49PM Pn")
Verify your settings:
* **Evaluation Name**: Auto-generated name
* **Your Trained Model**: Your customized model (with RAG)
* **Base Model**: Foundation model for comparison (without RAG)
* **Evaluation Type**: Model as Judge
Click **Start Comparison** to begin.
### Using the evaluation interface
![Screen Shot2025 11 24at3 22 00PM Pn]( "Screen Shot2025 11 24at3 22 00PM Pn")
![Screen Shot2025 11 24at3 22 00PM Pn]( "Screen Shot2025 11 24at3 22 00PM Pn")
The interface displays a side-by-side chat comparison:
* **Left panel**: Your trained model (with your data)
* **Right panel**: Base model (without your data)
**To test your models:**
1. Type your question in the input box
2. Send to both models simultaneously
3. Review responses in real-time
4. Scroll down to view automated evaluation results
### Understanding evaluation results
Each evaluation summary includes:
**Winner declaration**\
Shows which model provided the better response
**Factual grounding analysis**
* Response A (RAG): How well your model uses training data
* Response B (Base): Evaluation of the unenhanced model
**Key differences**\
Highlights why one response outperformed the other
**Winner rationale**\
Detailed explanation of the judge's decision
### Evaluation criteria
The AI judge evaluates responses based on:
* Factual accuracy from source material
* Proper use of grounding and training data
* Relevance to the question
* Completeness and clarity
Grounded responses using your training data consistently outperform speculative answers.
### Interpreting your results
**Your model wins**\
Your customization is working effectively. Training data is being used properly, and the model is ready for this use case.
**Base model wins**\
A knowledge gap has been identified. Add more training data on this topic and continue refinement.
**Mixed results**\
Partial success indicates you should add data for questions where your model underperformed and continue testing.
### Best practices
**Testing strategy:**
* Ask 10-15 diverse questions minimum
* Test scenarios where your data should provide an advantage
* Include difficult and edge cases
* Review "Past Results" to track improvement over time
**After evaluation:**
1. Identify patterns in wins and losses
2. Add training data to address knowledge gaps
3. Re-test to verify improvements
4. Iterate continuously
### Tips for success
* Grounded responses always outperform speculation
* Losses reveal where to add more training data
* Test regularly as you add new content
* Use realistic queries your actual users would ask
# Glossary
Source: https://docs.4minds.ai/glossary
Key terms and definitions for understanding 4MINDS AI platform
**Ghost Weights™**: Systematic assessments that measure model performance against specific criteria or benchmarks. Evaluations help you understand accuracy, consistency, and quality across different types of queries. The 4MINDS platform supports automated testing, human review, and model judging to ensure your AI meets your organization's standards before and during deployment.
**Ground Truth**: The verified correct answer used to evaluate model performance in question-and-answer pairs. In 4MINDS, ground truths are generated directly from your uploaded data rather than generic public datasets. This ensures evaluations test how well your model performs on your actual business context and proprietary knowledge, not just on industry-standard benchmark datasets like MMLU (Massive Multitask Language Understanding) or SQuAD (Stanford Question Answering Dataset).
**Synthesis Graph™**: 4MINDS' technology that automatically organizes your documents into intelligent, hierarchical knowledge structures that function as attention layers within the model architecture. Unlike approaches that only modify weights, Synthesis Graph™ works at the transformer level to understand how your information connects and routes queries efficiently across large-scale data, surfacing insights and relationships you might otherwise miss. It stays current as your information grows and changes.
**Reflex Router™**: 4MINDS' intelligent routing system that automatically determines the best way to answer each query. You get the most accurate responses without configuring anything. Reflex Router™ handles the complexity behind the scenes, ensuring fast answers for simple questions and deep insights for complex ones.
**Inline Tuning™**: A 4MINDS' platform feature enabling continuous model adaptation during active use. Your models get smarter as you use them, learning from interactions without requiring manual updates or service interruptions. Your models improve continuously without taking your system offline for updates.
***Note: Terms marked with ™ are proprietary 4MINDS technologies.***
**Evals (Evaluations)**: Systematic assessments that measure model performance against specific criteria or benchmarks. Evaluations help you understand accuracy, consistency, and quality across different types of queries. The 4MINDS platform supports automated testing, human review, and model judging to ensure your AI meets your organization's standards before and during deployment.
**Prompt**: The input text or question you provide to a model. Effective prompts help the model understand what you're asking and generate better responses. The 4MINDS platform uses your prompts along with context from your datasets to deliver accurate, relevant answers.
**SYMI**: 4MINDS' agentic AI layer — an autonomous workflow engine that sits on top of your trained models and acts on your behalf across email, CRM, and other connected systems. SYMI runs on its own base model (Qwen 3.6) and can route requests to any 4MINDS model you have trained. It triggers automatically on incoming events, runs scheduled jobs, and pushes outputs to external systems without manual intervention.
**Sub-Agent**: A specialized agent deployed within SYMI for a specific task. Each sub-agent has a label, a task prompt (which defines its role and behavior), and a model routing assignment. Sub-agent task prompts are where you define role-specific context — e.g. *"You are a cybersecurity analyst working in a SOC"* — as opposed to Personas, which control tone only.
**Model Router**: The component in SYMI that selects which 4MINDS model is queried for a given task. By default, SYMI uses its Qwen 3.6 base model; the router can direct requests to any trained model in your workspace, and per-sub-agent routing is supported for specialized workflows.
**Reasoning Log**: A summary-level audit trail of the steps SYMI took during a session — which models were queried, which sub-agents were invoked, key decision points, and what was pushed to external systems. Powered by Constellation, 4MINDS' structured cognitive reasoning system. Useful for debugging workflows and demonstrating reasoning transparency.
**Base Model**: The underlying foundation model selected for personalization within the 4MINDS platform. Examples include Gemma-12B, Nemotron-70B, and Qwen-32B. Choosing the right base model affects speed, accuracy, and capability—4MINDS offers multiple options so you can balance performance with your specific needs. The base model provides core language capabilities before Ghost Weights™ adaptation.
**Context Window**: The maximum amount of text (measured in tokens) a model can process in a single interaction. Context windows determine how much conversation history, document content, or retrieved information can inform each response. Larger context windows enable more comprehensive analysis and better understanding of complex queries. Common examples include 8K, 32K, or 128K tokens.
**Fine-tuning**: The process of adapting a pre-trained model to perform specific tasks or match particular domains by training on targeted datasets. Traditional fine-tuning updates all or most model parameters, making it resource-intensive and requiring service downtime.
**Inference**: The process of generating outputs from a trained model in production. Inference occurs when a deployed model processes queries and produces responses for actual use cases. Faster inference means quicker answers and better user experience.
**Knowledge Graph**: A structured representation of information as entities (nodes) and relationships (edges) that maps how concepts connect. Traditional knowledge graphs are often manually curated or require rigid schemas, making them static and labor-intensive to maintain. They typically struggle to scale beyond limited domains and can't efficiently route queries across large information spaces, often requiring manual configuration to determine which parts of the graph are relevant for specific questions. 4MINDS' proprietary implementation is called Synthesis Graph™—see above.
**LoRA (Low-Rank Adaptation)**: A parameter-efficient fine-tuning technique that updates a subset of model weights through low-rank matrix decomposition. LoRA operates at the weight level, making it more efficient than full fine-tuning but still requiring service downtime for model updates and lacking support for continuous adaptation. It doesn't address how models process and route information at the transformer layer.
**Multimodal**: AI systems capable of processing and generating multiple types of content beyond text, such as images, audio, video, and documents. Multimodal models can understand relationships across different content types—for example, answering questions about images or extracting information from visual documents. Traditional multimodal implementations often struggle with maintaining context across modalities and require separate processing pipelines for each content type, adding complexity to deployment and maintenance.
**Parameters**: The learned weights within a neural network that determine model behavior. Parameter count (e.g., 12B for 12 billion parameters) generally indicates model capability, though architecture efficiency varies. 4MINDS' Ghost Weights™ delivers better performance by training smarter—adapting less than 5% of parameters—rather than requiring massive parameter counts.
**Quantization**: A technique that reduces model size and speeds up inference by converting high-precision numerical values to lower-precision formats. Quantization makes models more efficient and cost-effective to run, enabling faster responses with less computational resources. The tradeoff is that aggressive quantization can impact model accuracy. Modern quantization methods like AWQ (Activation-Aware Weight Quantization) minimize accuracy loss while maximizing efficiency gains.
**RAG (Retrieval-Augmented Generation)**: An architecture pattern that retrieves relevant information from documents before generating responses, ensuring answers are grounded in actual content rather than the model's general knowledge. Traditional RAG implementations rely primarily on vector similarity matching, which excels at finding documents with matching keywords but struggles with contextual understanding, multi-hop reasoning, and discovering relationships between concepts. This can lead to missed insights when answers require connecting information across multiple sources or understanding how ideas relate.
**Token**: The fundamental unit of text processing in language models. A token typically represents a word, part of a word, or punctuation mark. Token counts determine context window limits and affect processing speed and cost.
**Training**: The process of teaching a model to perform specific tasks by exposing it to examples and data. Training can range from initial model development to ongoing adaptation. In the 4MINDS platform, Ghost Weights™ and Inline Tuning™ enable efficient, continuous training without disrupting your operations. In 4MINDS, training a model involves three key steps: creating a model, feeding it a dataset, and iteratively refining it through testing in the Playground where you can add more data as you discover knowledge gaps.
**Vector Search**: A search technique that finds information based on semantic similarity rather than exact keyword matches. Vector search converts text into numerical representations (embeddings) and identifies content with similar meanings. While effective for basic retrieval, vector search alone misses contextual relationships between information, which is why the 4MINDS platform combines it with Synthesis Graph™ for more intelligent results.
***
Can't find a term? [Contact our support team](mailto:support@4minds.ai) or suggest additions to this glossary.
# Gong
Source: https://docs.4minds.ai/gong
## Overview
The 4MINDS platform integrates with **Gong**, allowing you to import conversation intelligence data directly into your 4MINDS datasets. You can pull in call recordings, transcripts, user profiles, activity statistics, scorecards, keyword trackers, and CRM data — all converted into readable text files for analysis.
***
## Getting Started
### Prerequisites
* A Gong account with API access enabled
* Either a **Gong API key pair** (Access Key + Access Key Secret) or an **OAuth-enabled Gong account**
* Appropriate Gong permissions for the data types you want to access (see [Gong API Scopes](#gong-api-scopes) below)
### Connecting Your Gong Account
The Gong integration supports two authentication methods: **API Token** and **OAuth**.
#### Option A: API Token
1. Open **Integrations** from the main navigation bar in 4MINDS.
2. Find the **Gong** integration and click **Connect**.
3. Select the **API Token** tab.
4. Enter your **Access Key** and **Access Key Secret**.
* To find these: log in to Gong, go to **Company Settings** > **API**, and create a new API key if you don't have one.
5. Click **Test Connection** to verify your credentials.
6. Click **Save Credentials** to complete the setup.
> **Note:** The Access Key Secret is only shown when creating a new API key in Gong. Store it securely — you cannot retrieve it later.
#### Option B: OAuth
1. Open **Integrations** from the main navigation bar in 4MINDS.
2. Find the **Gong** integration and click **Connect**.
3. Select the **OAuth** tab.
4. Click **Connect with Gong**. A popup window will open.
5. Log in to your Gong account and authorize 4MINDS when prompted.
6. The popup will close automatically once authorization is complete.
> **Note:** OAuth uses secure OAuth 2.0 authentication. Your Gong password is never stored by 4MINDS. Access tokens are encrypted and refreshed automatically.
### Disconnecting
To disconnect your Gong account, open **Integrations** from the main navigation bar, find Gong, and click **Disconnect**. This removes your stored credentials and revokes access.
***
## Available Data Types
Unlike file-based integrations, Gong provides **data categories** that are fetched from the Gong API and converted into readable text files. Each data type produces a `.txt` file with a descriptive header and structured records.
| Data Type | Description | Example Output |
| ----------------------- | ------------------------------------------------------------------------------------------------ | ---------------------------------------------- |
| **Calls** | Call recordings and metadata — title, date, duration, direction, attendees, topics, and trackers | `gong_calls_20260409_120000.txt` |
| **Transcripts** | Full call transcripts with timestamped speaker turns | `gong_transcripts_20260409_120000.txt` |
| **Users** | Gong user profiles — name, email, title, role, and account status | `gong_users_20260409_120000.txt` |
| **User Stats** | Daily user activity and interaction statistics | `gong_user-stats_20260409_120000.txt` |
| **Scorecards** | Call scoring and evaluation template definitions | `gong_scorecards_20260409_120000.txt` |
| **Answered Scorecards** | Completed call scores and evaluations | `gong_answered-scorecards_20260409_120000.txt` |
| **Libraries** | Saved call libraries and snippet collections | `gong_libraries_20260409_120000.txt` |
| **Trackers** | Keyword and topic trackers being monitored | `gong_trackers_20260409_120000.txt` |
| **CRM Deals** | Deal data synced from your CRM into Gong | `gong_crm-deals_20260409_120000.txt` |
| **CRM Contacts** | Contact records synced from your CRM into Gong | `gong_crm-contacts_20260409_120000.txt` |
| **CRM Accounts** | Account/company data synced from your CRM into Gong | `gong_crm-accounts_20260409_120000.txt` |
### Import Limits
Each data type imports up to **100 records** per request. For data types with pagination (calls, transcripts, users, etc.), the platform fetches multiple pages until reaching the limit.
***
## Importing Data
### Creating a Dataset with Gong Data
1. Create a new dataset (or edit an existing one).
2. Select **Gong** as a data source.
3. The platform checks your Gong connection. If not connected, you will be prompted to set up the connection first.
4. A list of all available data types is displayed with checkboxes.
5. Use **Select All** to choose all data types, or select individual types.
6. Click **Add** to stage the selected data types for import.
7. The staged items appear in your attached files list with estimated file sizes.
8. Complete the dataset creation to trigger the import.
### Combining with Other Sources
Gong data can be combined with files from other sources in the same dataset. For example, you can import Gong call transcripts alongside Google Drive documents or uploaded files.
***
## Dataset Sync
Once you have imported Gong data into a dataset, you can set up **automatic syncing** to keep your dataset up to date with the latest data from Gong.
### How Sync Works
* Each Gong data type is synced independently. For example, you can sync Calls and Transcripts without syncing Users.
* On each sync cycle, the platform fetches the latest data from Gong, converts it to text, and updates the corresponding file in your dataset.
* The text file is fully regenerated on each sync (not appended), so it always reflects the current state of your Gong data.
### Sync Options
* **Sync specific data types:** Choose which data types to keep in sync.
* **Sync all:** Automatically sync all data types that were originally imported.
### Configuring Sync
When creating a dataset with Gong data:
1. Toggle **Dataset Sync** on before completing the dataset.
2. Choose the sync frequency (e.g., daily, weekly).
3. Optionally deselect specific data types from syncing.
***
## Text File Format
Each imported file includes a descriptive header followed by structured records. Here is an example of a Calls import:
```text theme={null}
Gong Calls Import — 5 calls
Imported: April 09, 2026 at 14:00 UTC
This file contains call recordings and metadata imported from the organization's Gong account.
Each call includes the title, date, duration, direction, attendees, topics, and trackers.
----------------------------------------
=== CALL ===
Title: Q2 Pipeline Review
Date: 2026-04-08 10:30:00 UTC
Duration: 45m 00s
Direction: Outbound
Attendees:
- Alice Smith (Internal)
- Bob Johnson (External)
Topics: Pricing, Timeline
Trackers: Competitor Mention
=== CALL ===
Title: Product Demo — Acme Corp
...
```
***
## Gong API Scopes
Different data types require different API permission scopes in Gong. If your API key or OAuth token does not have the required scope, that data type will return a permission error and be skipped — other data types will still import successfully.
| Data Type | Required Gong Scope |
| ------------------------------- | ------------------------------ |
| Calls | `api:calls:read:extensive` |
| Transcripts | `api:calls:read:transcript` |
| Users | `api:users:read` |
| User Stats | `api:stats:user-actions` |
| Scorecards | `api:settings:scorecards:read` |
| Answered Scorecards | `api:stats:scorecards` |
| Libraries | `api:library:read` |
| Trackers | `api:settings:trackers:read` |
| CRM Deals / Contacts / Accounts | `api:crm:read` |
> **Tip:** If you see a "permission" error for a specific data type, check your Gong API key scopes in **Gong** > **Company Settings** > **API** and ensure the required scope is enabled.
***
## Navigation & UI Features
### Data Type Selection
When importing from Gong, each data type is displayed as a card with an icon, name, and description. Select data types using the checkboxes.
### Select All / Deselect All
Use the **Select All** button at the top of the data type list to quickly select or deselect all types.
### Size Estimation
When you add Gong data types to your dataset, the platform fetches a preview to estimate the file size. The estimated size is displayed in the attached files list.
***
## Troubleshooting
| Issue | Solution |
| ------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **"No Gong connection found"** | Open Integrations from the main nav and connect your Gong account using API Token or OAuth. |
| **Permission error for a data type** | Your Gong API key may not have the required scope. Check Gong > Company Settings > API and enable the needed permissions. See the [scopes table](#gong-api-scopes) above. |
| **401 Unauthorized error** | Your credentials may have expired or been revoked. Disconnect and reconnect your Gong account. |
| **No data returned for a data type** | The data type may be empty in your Gong account (e.g., no calls recorded, no CRM data synced). |
| **User Stats returns no data** | User Stats requires a date range. The platform defaults to the last 30 days. If your account is new, there may be no activity data yet. |
| **Connection expired** | For OAuth connections, the platform automatically refreshes tokens. If issues persist, disconnect and reconnect. For API Token connections, verify your key has not been revoked in Gong. |
| **Popup blocked during OAuth** | Enable popups for the 4MINDS site in your browser settings, then try again. |
| **Import takes a long time** | Some data types (calls, transcripts) may take longer to fetch if your Gong account has many records. The import fetches up to 100 records per data type. |
***
## Security & Privacy
* **OAuth 2.0:** OAuth connections use industry-standard OAuth 2.0. Your Gong credentials are never stored by 4MINDS.
* **Encrypted tokens:** All access tokens and API key secrets are encrypted at rest.
* **Automatic token refresh:** OAuth tokens are refreshed automatically before they expire, so you stay connected without re-authenticating.
* **Per-user access:** Each user connects their own Gong account. Your Gong connection is not shared with other users on the platform.
* **Data conversion:** Gong data is fetched in real time and converted to text files. Raw API responses are not stored — only the formatted text content is saved in your dataset.
***
## FAQ
**Q: What Gong data can I access?** A: You can access any data type that your Gong API key or OAuth token has permissions for. See the [data types table](#available-data-types) for the full list.
**Q: Can I import Gong data into an existing dataset?** A: Yes. You can add Gong data to both new and existing datasets, and combine it with data from other sources.
**Q: How often does dataset sync run?** A: Sync frequency depends on your dataset's sync configuration. You can also trigger a manual sync at any time.
**Q: What happens to my data if I disconnect my Gong account?** A: Previously imported data remains in your datasets. However, automatic syncing will stop, and you will not be able to import new data from Gong until you reconnect.
**Q: Can I choose which data types to sync?** A: Yes. When setting up dataset sync, you can select or deselect individual data types. Only selected types will be refreshed on each sync cycle.
**Q: Are call audio recordings imported?** A: No. The integration imports call metadata (title, duration, attendees, etc.) and transcripts as text. Audio files are not downloaded or stored.
**Q: How much data is imported per data type?** A: Each data type imports up to 100 records. For calls and transcripts, this means the 100 most recent entries. For CRM data, up to 100 objects per type.
**Q: Can I connect multiple Gong accounts?** A: Each user account on 4MINDS supports one Gong connection at a time. To switch accounts, disconnect the current one and connect with a different Gong account.
# HubSpot
Source: https://docs.4minds.ai/hubspot
Connect 4MINDS to HubSpot to import CRM data for AI-powered analysis
## Overview
The 4MINDS platform integrates with **HubSpot**, allowing you to import CRM data directly into your 4MINDS models. You can pull in contacts, companies, deals, tickets, and other CRM objects — making them available for AI-powered querying, knowledge graph construction, and retrieval.
***
## Getting Started
### Prerequisites
* A HubSpot account with access to the CRM objects you want to import
* A 4MINDS account with access to the Integrations page
### Connecting Your HubSpot Account
1. Sign in to [4MINDS](https://app.4minds.ai).
2. Navigate to the **Integrations** page from the main navigation bar.
3. Locate the **HubSpot** integration card and click on it.
4. Click **Connect with HubSpot**. You will be redirected to HubSpot's OAuth authorization screen.
5. Sign in to HubSpot (or use an existing session) and approve the requested scopes.
6. Once authorized, you will be redirected back to 4MINDS and the connection will be established.
### Disconnecting
To disconnect your HubSpot account, open **Integrations** from the main navigation bar, find HubSpot, and click **Disconnect**. This removes your stored credentials and revokes access.
***
## OAuth Scopes
When you connect your HubSpot account, 4MINDS requests the following OAuth scopes:
| Scope | Description |
| ---------------------------- | -------------------------- |
| `crm.objects.contacts.read` | Read contact records |
| `crm.objects.companies.read` | Read company records |
| `crm.objects.deals.read` | Read deal records |
| `crm.objects.orders.read` | Read order records |
| `crm.objects.owners.read` | Read owner/user records |
| `crm.schemas.custom.read` | Read custom object schemas |
| `crm.import` | Import CRM data |
| `tickets` | Read ticket records |
| `files` | Access file attachments |
| `oauth` | OAuth authentication |
> **Note:** You must have sufficient permissions in your HubSpot account for each scope. If a scope is not available for your HubSpot plan, the corresponding data type may not be importable.
***
## Importing Data
### Creating a Model with HubSpot Data
1. Navigate to the **Models** page and click **Create Model**.
2. Provide a **name** and **use case** for the model (e.g., "HubSpot CRM Test" / "Analyze imported CRM data"). Press **Next**.
3. On the **Upload Data** step, select the **Integrations** option.
4. Scroll down to the **Enterprise** group.
5. Choose **HubSpot** from the connected integrations list.
6. Select the objects you'd like to import (contacts, companies, deals, tickets, etc.).
7. Confirm the import and create the model.
Once processing completes, the dataset will be available for AI-powered querying, knowledge graph construction, and retrieval on the **Models** page.
### Querying Your Data
1. Find the model you created on the **Models** page.
2. Click the **Run** button.
3. On the chat interface, send a prompt or query to get AI-generated responses about your HubSpot data.
### Combining with Other Sources
HubSpot data can be combined with files from other sources in the same model. For example, you can import HubSpot contacts alongside Google Drive documents or uploaded files.
***
## Available Data Types
The following HubSpot CRM objects can be imported into 4MINDS:
| Data Type | Description |
| ------------- | -------------------------------------------------------------------------------------- |
| **Contacts** | Contact records — names, emails, phone numbers, lifecycle stage, and custom properties |
| **Companies** | Company records — name, domain, industry, revenue, and associated contacts |
| **Deals** | Deal records — deal name, stage, amount, close date, and pipeline information |
| **Tickets** | Support ticket records — subject, status, priority, and associated contacts |
| **Orders** | Order records — order details and associated deal/contact information |
| **Owners** | HubSpot user/owner records — name, email, and team assignment |
***
## Troubleshooting
| Issue | Solution |
| --------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| **"No HubSpot connection found"** | Open Integrations from the main nav and connect your HubSpot account via OAuth. |
| **OAuth authorization fails** | Ensure you have admin or sufficient permissions in your HubSpot account to authorize third-party apps. |
| **Missing data types** | Some CRM objects may not be available depending on your HubSpot plan (Starter, Professional, Enterprise). |
| **No data returned** | The selected object type may be empty in your HubSpot account (e.g., no deals created yet). |
| **Connection expired** | The platform automatically refreshes OAuth tokens. If issues persist, disconnect and reconnect your HubSpot account. |
| **Popup blocked during OAuth** | Enable popups for the 4MINDS site in your browser settings, then try again. |
***
## Security & Privacy
* **OAuth 2.0:** Connections use industry-standard OAuth 2.0. Your HubSpot credentials are never stored by 4MINDS.
* **Encrypted tokens:** All access tokens are encrypted at rest.
* **Automatic token refresh:** OAuth tokens are refreshed automatically before they expire, so you stay connected without re-authenticating.
* **Per-user access:** Each user connects their own HubSpot account. Your HubSpot connection is not shared with other users on the platform.
***
## FAQ
**Q: What HubSpot data can I access?** A: You can access contacts, companies, deals, tickets, orders, and owners — any CRM object that your HubSpot account has permissions for.
**Q: Can I import HubSpot data into an existing model?** A: Yes. You can add HubSpot data to both new and existing models, and combine it with data from other sources.
**Q: What happens to my data if I disconnect my HubSpot account?** A: Previously imported data remains in your models. However, you will not be able to import new data from HubSpot until you reconnect.
**Q: Do I need a specific HubSpot plan?** A: The integration works with HubSpot Free, Starter, Professional, and Enterprise plans. However, some CRM objects (like custom objects) may only be available on higher-tier plans.
**Q: Can I connect multiple HubSpot accounts?** A: Each user account on 4MINDS supports one HubSpot connection at a time. To switch accounts, disconnect the current one and connect with a different HubSpot account.
# Add Integrations & Data Sources
Source: https://docs.4minds.ai/integrations
Connect your 4MINDS platform to external data sources like HuggingFace Hub and Databricks. Import datasets seamlessly, track progress, and validate connections: all from a unified interface.
## Overview
The 4MINDS platform integrates with popular data platforms to streamline your workflow. Instead of manually downloading and uploading files, connect directly to your existing data sources and import what you need.
**Some integrations require setup on the provider's side before they'll work in 4MINDS.** For example, foundation models on Amazon Bedrock must be requested and approved in your AWS account before you can use them in 4MINDS. Integrations with provider-side prerequisites include a **Provider prerequisites** section in their entry below — complete those steps in the provider's console *before* connecting in 4MINDS.
## Supported Integrations
![Screen Shot2025 11 25at7 01 32PM Pn]()
![Screen Shot2025 11 25at7 01 32PM Pn]()
### Featured
#### HubSpot
Connect to HubSpot CRM to access customer data, conversations, and marketing content for enhanced AI insights. Import contact records, deal information, email interactions, and content assets directly into your 4MINDS workspace to power customer-aware AI experiences. See the [HubSpot integration guide](/hubspot) for setup details.
#### Amazon Bedrock
Connect to Amazon Bedrock to create models from AWS-managed foundation model endpoints. Access AWS's fully managed service for foundation models and integrate them directly into your 4MINDS workflows. See the [Amazon Bedrock integration guide](/bedrock) and the [AWS Integrations guide](/aws-integrations) for setup details.
**Provider prerequisites**
Before connecting Bedrock to 4MINDS, complete the following in your AWS account:
* Request access to the foundation models you want to use in **Amazon Bedrock → Model access**.
* Wait for each model's status to show **Access granted** (some models require AWS approval).
* Confirm your IAM role has `bedrock:InvokeModel` permission for the granted models.
See AWS's [Manage access to Amazon Bedrock foundation models](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html) for the full process.
#### Databricks
Link your Databricks workspace to pull data from tables, notebooks, and files. This integration lets you leverage existing data pipelines without duplicating storage or creating manual export processes. See the [Databricks integration guide](/databricks) for setup details.
### Microsoft
#### Microsoft 365
Connect to Microsoft 365 for seamless access to emails, calendars, OneDrive files, Teams messages, and OneNote. Integrate your Microsoft 365 ecosystem to leverage organizational knowledge and collaboration data.
#### Microsoft Foundry
Connect to Microsoft Foundry to create models from external models you've deployed on the Foundry platform. Leverage your existing Foundry deployments to power AI experiences on 4MINDS without redeploying infrastructure. See the [Microsoft Foundry integration guide](/microsoft-foundry) for setup details.
#### Microsoft SharePoint
Connect to Microsoft SharePoint to access documents and files stored in your organization's SharePoint sites. Import data directly from SharePoint libraries, maintaining your existing document management workflows.
#### Azure Blob Storage
Connect to Azure Blob Storage to access your cloud-stored datasets and files. Import data directly from your Azure containers without manual downloads, maintaining your existing cloud storage infrastructure.
**Sync scope:** The integration does **not** sync every file your credentials can reach. It syncs everything inside the **scope you connected** — current files, future files, and any new subfolders added later.
**How the sync scope is determined when staging files:**
| What you did when staging | "All folders and files" | Resulting sync scope |
| ----------------------------------------------------------------------- | ----------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| Staged files from **My Drive root** | ✅ Checked | Your entire My Drive, recursively — every folder, subfolder, and file, plus anything added later. |
| Staged files from a **specific subfolder** (e.g., `My Drive/Projects/`) | ✅ Checked | Scope expands back up to your **full My Drive**. Leave the box unchecked if you only want the subfolder. |
| Staged files from a **specific folder** | ❌ Unchecked | That folder and all of its subfolders recursively, including new files added later. This is typically what users want. |
If you select a specific folder, **leave "All folders and files" unchecked** to keep the sync scoped to that folder and its subfolders.
#### Microsoft Fabric
Connect to Microsoft Fabric to access your unified analytics platform data. Browse and import directly from your Fabric Lakehouses (files) and Warehouses (tables), leveraging Microsoft's end-to-end analytics solution for seamless data integration. See the [Microsoft Fabric integration guide](/microsoft-fabric) for setup details.
#### Microsoft 365 Copilot
Install the **4MINDS agent** inside Microsoft 365 Copilot to chat with your custom 4MINDS models from anywhere Copilot appears — Teams, Office on the web, and the Office desktop app for Windows. Ask questions and get answers grounded in your organization's own documents, specifications, and playbooks. See the [Microsoft 365 Copilot guide](/microsoft-365-copilot) for install and setup details.
### Google
#### Google Workspace
Connect to Google Workspace (GSuite) to access organizational data across Gmail, Drive, Calendar, and Docs. Import emails, files, events, and documents directly into your 4MINDS workspace to build AI models with broad team and organizational context.
#### Google Vertex AI
Connect to Google Vertex AI to create models on the 4MINDS platform from your existing Vertex AI deployments. Leverage your trained Vertex AI models and endpoints to power AI experiences without duplicating model infrastructure.
#### Google Drive
Connect to Google Drive to access documents, spreadsheets, presentations, and files stored in your Drive. Import data directly from your personal or shared drives, maintaining your existing Google Workspace workflows.
#### Google Cloud Storage
Connect to Google Cloud Storage to access your cloud-stored datasets and files. Import data directly from your GCS buckets without manual downloads, maintaining your existing Google Cloud storage infrastructure.
### Amazon
#### Amazon S3
Connect to Amazon S3 to access your cloud-stored datasets and files. Import data directly from your S3 buckets without manual downloads, leveraging your existing AWS storage infrastructure.
**Setup requirements:**
* An **IAM role** in your AWS account that 4MINDS can assume (via IAM Role Federation or Amazon Cognito).
* The role must have, at minimum, **`s3:GetObject`** permission on the buckets you want to expose. Scope the policy to specific bucket ARNs (e.g., `arn:aws:s3:::my-bucket/*`) so only allowed buckets are accessible.
For step-by-step instructions — including the exact IAM permissions policy, trust policy, and how to attach the role in 4MINDS — see the [AWS Integrations guide](/aws-integrations).
#### AWS Lake Formation
Connect to AWS Lake Formation to access governed data lakes with fine-grained access controls. Import datasets directly from your Lake Formation-managed catalogs, leveraging AWS's centralized data governance for secure, compliant data integration. See the [AWS Integrations guide](/aws-integrations) for setup details.
#### Amazon SageMaker
Connect to Amazon SageMaker to create models on the 4MINDS platform from your existing SageMaker models. Leverage your trained SageMaker models and endpoints to power AI experiences without duplicating model infrastructure. See the [AWS Integrations guide](/aws-integrations) for setup details.
#### Amazon RDS
Connect to Amazon RDS to browse your managed relational databases (PostgreSQL, MySQL, MariaDB, Aurora) and import tables as datasets. 4MINDS discovers instances via IAM and opens a live SQL connection to each instance only when you choose to browse its databases — master DB credentials stay in your browser and are never persisted. See the [Amazon RDS integration guide](/rds) for setup details.
### Enterprise
#### Databricks
Link your Databricks workspace to pull data from tables, notebooks, and files. This integration lets you leverage existing data pipelines without duplicating storage or creating manual export processes. See the [Databricks integration guide](/databricks) for setup details.
#### Snowflake
Connect to Snowflake to access your data warehouse tables and views. Import structured data directly from Snowflake without manual exports, leveraging your existing data warehouse infrastructure.
#### dbt
Connect to dbt Cloud to import your project's metadata — model definitions, sources, tests, exposures, and account structure — for AI-powered analysis. Connect read-only via OAuth or with a service token. See the [dbt integration guide](/dbt) for setup details.
#### HubSpot
Connect to HubSpot CRM to access customer data, conversations, and marketing content for enhanced AI insights. Import contact records, deal information, email interactions, and content assets directly into your 4MINDS workspace to power customer-aware AI experiences. See the [HubSpot integration guide](/hubspot) for setup details.
#### Salesforce
Connect to Salesforce CRM to access leads, accounts, contacts, opportunities, and customer data. Import your sales pipeline, customer records, and business analytics directly into 4MINDS to build AI models with comprehensive CRM knowledge.
### General Use
#### HuggingFace Hub
Connect to the HuggingFace Hub to access thousands of public datasets and models. Browse available resources, preview metadata, and import datasets directly into your 4MINDS workspace.
Parquet files are not supported when importing datasets from Hugging Face. Please use CSV, JSON, or other supported formats instead.
#### Box
Connect to Box to access your cloud-stored documents and files. Import data directly from your Box folders without manual downloads, leveraging your existing Box storage infrastructure for seamless collaboration and file management.
#### Slack
Connect to Slack to access workspace messages, channels, and conversation history. Import team communications and knowledge shared across channels to build AI models that understand your organization's context and discussions.
#### Dropbox
Connect to Dropbox to access your cloud-stored files and folders. Import documents and data directly from your Dropbox account without manual downloads, leveraging your existing file storage infrastructure.
### Additional Integrations
#### CoreWeave Storage
Connect to CoreWeave Storage to access your datasets stored on CoreWeave's high-performance cloud infrastructure. Import data directly from your CoreWeave storage without manual transfers.
#### NetApp
Connect to NetApp storage systems to access your enterprise data and files. Import data directly from NetApp storage volumes, leveraging your existing enterprise storage infrastructure for seamless data integration.
#### Supabase
Connect to Supabase to access your PostgreSQL databases, storage buckets, and real-time data. Import tables and files directly from your Supabase projects, leveraging your existing backend infrastructure for seamless data integration.
#### Splunk
Connect to Splunk to access machine data, logs, and analytics. Import search results and indexed data directly from your Splunk deployment to build AI models with operational intelligence and observability insights.
#### ServiceNow
Connect to ServiceNow to access IT service management data, incident records, and knowledge base articles. Import tickets, workflows, and operational data directly into 4MINDS to build AI models with enterprise IT knowledge.
#### Gong
Connect to Gong to import revenue intelligence and sales conversation data — including call recordings, transcripts, user profiles, activity stats, scorecards, and CRM data — directly into your 4MINDS datasets. See the [Gong integration guide](/gong) for setup details.
## How it Works
1. **Connect** - Authenticate with your external platform using API keys or OAuth
2. **Browse** - Explore available datasets and resources from the connected source
3. **Import** - Select the data you want and initiate the import process
4. **Track** - Monitor import progress with real-time status updates
5. **Validate** - Confirm successful connections and data integrity before training
## Upload Size Limit
* The Enterprise tier allows up to **2 GB per batch**, but uploads are backed by Azure Blob Storage, which enforces a **100 MB limit per request**. In practice, this means each upload request is capped at 100 MB — anything larger must be split into multiple 100 MB batches.
* This applies to single files, multiple files, and integration datasets. A progress bar will display the total upload size.
* To upload more data, simply reopen the dataset and upload the next 100 MB batch.
There is no limit on the overall dataset size — only on each individual upload request. The Enterprise tier permits up to 2 GB total per upload session, but it must be delivered in batches of 100 MB or less due to the Azure Blob Storage per-request limit.
## Managing Connections
All active integrations appear in your Control Center. You can:
* View connection status and last sync time
* Test connections to verify they're working
* Update credentials or permissions
* Disconnect sources you no longer need
Once imported, datasets from external sources work exactly like uploaded files: they're processed through the same ETL and Graph engines to build your model's knowledge base.
## Automatic Data Synchronization (Rsync)
Keep your datasets current with automatic synchronization from connected integrations. When you create a dataset from an integration source (like a OneDrive folder), 4MINDS saves a manifest tracking the files in that location. Each time you log in, the platform checks your connected sources for new files and automatically fetches any additions to your dataset.
**How it works:**
1. Create a dataset from an integration source (e.g., a OneDrive folder)
2. 4MINDS stores a manifest of the files at creation time
3. On each login, the platform compares the current source contents against the manifest
4. New files are automatically retrieved and added to your dataset
This ensures your AI models always have access to the latest data without manual re-imports.
# What is 4MINDS
Source: https://docs.4minds.ai/introduction
4MINDS is an enterprise AI deployment platform that transforms how organizations deploy and adapt open-source language models. Instead of adding search tools to a fixed AI model, 4MINDS builds your business knowledge into the model itself. This means it keeps learning from your data without any downtime.
## How 4MINDS Works
The platform combines three proprietary technologies that work together to deliver a fundamentally different AI architecture. Instead of relying on separate, disconnected components, these technologies form an integrated system: the Synthesis Graph™ structures your knowledge into a dynamic network, Ghost Weights™ embed this knowledge directly into the model at the transformer parameter level, ensuring personalization becomes part of the model's core reasoning, and the Reflex Router™ intelligently directs queries to ensure fast, accurate responses. Together, they create an AI that adapts to your business in real-time while maintaining the speed and reliability you need.
## Get Started
Ready to build your first AI model? The Quick Start Guide walks you through creating an account, uploading your data, and training your first model in minutes.
Build and test your first AI model in minutes
Learn why 4MINDS delivers better results than traditional approaches
Explore the full range of Control Center functionality
# Key Advantages
Source: https://docs.4minds.ai/key-advantages
Most open-source enterprise AI platforms use retrieval-augmented generation layered on either vector databases or knowledge graph databases. Vector-based approaches embed your documents, perform similarity searches, and pass matching chunks to the model. This works well for simple lookup tasks and factual questions. The limitation emerges with complex queries that require reasoning across multiple pieces of information or discovering connections between concepts - vector similarity alone can't construct those insights.
> We built technologies that work together to deliver a fundamentally different AI architecture.
## **Why This Matters**
**You get answers you can trust.** Every response includes explainable reasoning paths through the knowledge graphs. You see exactly how the AI connected information across your data. No black box. No guessing why it said what it said.
**Your AI stays current without downtime.** Models update continuously as your operations evolve - incorporating new data, corrections, and feedback while staying live in production. The alternative is scheduled maintenance windows where your AI goes offline, or static models that become outdated.
**The model becomes yours.** Ghost Weights™ train actual parameters within the transformer, not bolt-on adapters. This means genuine integration with your enterprise knowledge, not surface-level customization stacked on someone else's foundation. The model is trained specifically for your operations and adapts continuously to your evolving needs.
**Reasoning beats retrieval.** Synthesis Graph™ discovers connections across domains that vector search alone cannot find. This means better answers to complex questions, insights that span multiple data sources, and AI that actually understands context rather than just matching keywords.
# Microsoft 365 Copilot
Source: https://docs.4minds.ai/microsoft-365-copilot
Bring your 4MINDS custom AI models into Microsoft 365 Copilot. Ask questions and get answers grounded in your enterprise data — inside Copilot in Teams, Office on the web, and the Office app for Windows.
## Overview
**4MINDS Assistant** is a Microsoft 365 Copilot Declarative Agent that surfaces your 4MINDS custom AI models inside Microsoft 365 Copilot. Ask a question in the Copilot chat pane and the agent routes it to a model fine-tuned on your organization's documents, specifications, playbooks, and integrations — returning an answer grounded in your own data, not a generic web summary.
The agent runs as a first-class M365 Copilot extension, so nothing new to install for the day-to-day user beyond the agent itself. Sign-in is one-time per user; after that, chat with the agent from wherever you already use Copilot.
**Direction:** This integration puts **4MINDS inside Copilot** — the reverse of most Microsoft integrations, which bring Microsoft data *into* 4MINDS. If you're looking to import content from Microsoft Fabric or Microsoft Foundry into your 4MINDS datasets, see the [Microsoft Fabric](/microsoft-fabric) or [Microsoft Foundry](/microsoft-foundry) guides instead.
***
## Where the agent appears
Once installed, **4MINDS** shows up in the Microsoft 365 Copilot agent list in every host where your organization has enabled Copilot:
| Host | How to reach the agent |
| --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Microsoft 365 Copilot Chat in Teams** | Open Copilot in Teams → click the **Agents** icon → select **4MINDS**. |
| **Office on the web** | Open the Copilot pane in Word, Excel, PowerPoint, or Outlook on the web → open the agent picker → select **4MINDS**. Also reachable from [m365.cloud.microsoft](https://m365.cloud.microsoft) directly. |
| **Office app for Windows** | Open the Copilot pane in the Office desktop app or Outlook for Windows → open the agent picker → select **4MINDS**. |
Mobile availability (iOS / Android) follows the same tenant rollout as the desktop and web surfaces.
***
## Prerequisites
Before installing the agent, confirm the following are in place:
* **A 4MINDS account** on the tenant you plan to connect. If your organization has SSO with 4MINDS enabled, this is the email you already use to sign in.
* **At least one 4MINDS model** available to your account. The agent can only answer from models you have access to — if your workspace has none trained yet, see the [Quickstart](/quickstart) to publish your first one.
* **A Microsoft 365 Copilot license** on the same Microsoft account. The agent surfaces inside Microsoft 365 Copilot; without a Copilot license, the agent won't appear in the picker.
* **Tenant admin approval** for the 4MINDS agent. Declarative Agents require admin consent for the first user who signs in from your tenant — see [Admin: approving the agent for your tenant](#admin-approving-the-agent-for-your-tenant) below.
**License clarification:** Using the 4MINDS Copilot agent requires **both** an active 4MINDS subscription (for the model inference) **and** a Microsoft 365 Copilot license (for the host). Neither one on its own is sufficient.
***
## Installing the agent
There are two install paths depending on how your tenant governs agents:
### Option 1: Install from the Microsoft 365 Copilot Store
For end-users on tenants where the store is open:
1. Open the **Microsoft 365 Copilot Store** — reachable from the agent picker in any Copilot surface, or directly at [m365.cloud.microsoft](https://m365.cloud.microsoft).
2. Search for **4MINDS**.
3. Click **Add** on the **4MINDS Copilot — Custom AI Assistant for Microsoft 365** listing.
4. Accept the permission prompt (the agent needs to call the 4MINDS API on your behalf to fetch model responses).
5. The agent is now available in the agent picker across every Copilot host — no page refresh needed.
### Option 2: Admin sideload for your tenant
For organizations that keep the Copilot store restricted, or that want to deploy the agent tenant-wide before it appears in each user's store, an admin can sideload the app package directly:
1. From your 4MINDS contact, obtain the **`appPackage.prod.zip`** artifact for the agent.
2. In the **Microsoft Teams Admin Center**, navigate to **Teams apps → Manage apps**.
3. Click **Upload new app → Upload**, and select the `appPackage.prod.zip` file.
4. Once the app appears in the list, open its detail page and use **Publish to the org's app catalog** so members can install it.
5. To push the agent out without waiting for users to add it manually, create or edit an **app setup policy** in **Teams apps → Setup policies** and add **4MINDS** to the **Installed apps** list for the policy.
6. Assign the policy to the user groups you want to have the agent by default.
**Tenant-wide install takes up to 24 hours to propagate** through Microsoft's app catalog. If the agent doesn't appear immediately after the sideload, check back the next day before troubleshooting.
***
## First-time sign-in
The first time a user opens the 4MINDS agent in Copilot, they'll be prompted to authenticate to their 4MINDS account:
1. Open Copilot in any host (Teams, Office on the web, Office for Windows).
2. Select **4MINDS** from the agent picker.
3. Type any message (for example, `hello`). The agent will respond with a sign-in card.
4. Click **Sign in** and complete the OAuth flow with your 4MINDS credentials in the popup window. If SSO is enabled for your tenant, this is the same identity provider you already use for 4MINDS.
5. Once the popup confirms the connection, return to Copilot and re-send your message — the agent will now respond with a grounded answer from your 4MINDS models.
The sign-in is stored per user and refreshed automatically; you'll only be prompted again if your 4MINDS token is revoked or your organization changes SSO providers.
***
## Using the agent
### Conversation starters
The agent ships with three suggested prompts you'll see when opening a fresh chat:
| Starter | What it does |
| -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **"What models do I have?"** | Enumerates the 4MINDS models available to your account. Use this before asking a substantive question if you want to pick a specific model. |
| **"Apply our internal formula"** | Example of a task-based prompt — asks a domain model (finance playbook, engineering spec, etc.) to produce a concrete artifact like an Excel formula or a code snippet. |
| **"Pull a value from our docs"** | Example of a lookup-style prompt — asks the agent to retrieve a specific fact from your indexed documents. |
You can of course type any question; the starters are just examples of the shape.
### Switching between models
If your 4MINDS workspace has multiple models (for example, a general knowledge model, a finance model, and a security model), you can point the agent at a specific one:
1. Ask **"What models do I have?"** — the agent will list them by name.
2. In your next message, say something like **"Use the finance model to explain the Q3 margin adjustment,"** or reference the model by name inline.
The agent tracks the chosen model for the rest of the conversation until you switch again or start a new chat.
### Multi-turn context
The agent maintains conversation context automatically — you can reference earlier answers in the same chat with pronouns ("expand on that", "show me the numbers behind it") and the model will pick them up. Starting a new chat resets the context; use the **New chat** button in the Copilot pane when you want a clean slate.
### Styling of responses
The agent is instructed to lead with the grounded answer, so responses skip generic AI disclaimers. When you ask for code (Excel formulas, VBA, SQL, etc.), the agent formats the code in a fenced block so you can copy it directly into your document.
***
## Admin: approving the agent for your tenant
The 4MINDS agent's action plugin calls the 4MINDS API on behalf of the signed-in user. Microsoft 365 requires an **Azure AD admin consent** for this on the first install per tenant.
1. When the first user in your tenant signs in to the agent, Microsoft will surface an **Admin approval required** screen if consent hasn't been granted yet.
2. A Global Admin or Cloud Application Administrator opens the **Enterprise applications** view in the Azure portal, finds the **4MINDS Copilot** application, and clicks **Grant admin consent for \[tenant]**.
3. After consent is granted, every user in the tenant can sign in normally — the approval step is one-time per tenant, not per user.
**Data governance:** The agent only passes the user's message + the ID of the chosen 4MINDS model to the 4MINDS API. No Microsoft 365 content (documents, emails, calendar entries) is read by the agent — it operates on the models and datasets you've already published in 4MINDS.
### Restricting who can install the agent
If you want to limit the 4MINDS agent to a specific department or pilot group:
1. In **Teams Admin Center → Teams apps → Permission policies**, create or select a policy that blocks or allows the **4MINDS** app.
2. Assign the policy to the user groups you want to gate.
3. Users outside those groups won't see the agent in the store or the picker.
***
## Troubleshooting
| Symptom | Cause | Fix |
| -------------------------------------- | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Agent doesn't appear in the picker | Copilot license missing, or the app hasn't been added to your account. | Confirm the user has a Microsoft 365 Copilot license, then reinstall from the store or ask your admin to add the agent via the sideload path. |
| Agent asks to sign in on every message | The OAuth token isn't being persisted — usually a browser third-party cookie block during the popup. | Have the user complete the sign-in flow in a browser session with third-party cookies allowed for `login.microsoftonline.com`, or use the Office desktop app / Teams client where cookie policy is more permissive. |
| Response says "no models available" | The signed-in user has a 4MINDS account but no models are visible to them. | Check the user's 4MINDS workspace — either publish a model or invite them to a workspace that has one. |
| "Admin approval required" on sign-in | The tenant hasn't granted admin consent yet. | See [Admin: approving the agent for your tenant](#admin-approving-the-agent-for-your-tenant) above. |
| Agent appears but is greyed out | Tenant admin has restricted the agent via a permission policy. | Ask your Microsoft 365 admin to add you to a permission policy that allows **4MINDS**. |
| Response feels generic / not grounded | The user hasn't selected a specific model, or the chosen model isn't fine-tuned on the topic. | Ask **"What models do I have?"** and switch to the model that owns the data for your question. |
If none of the above resolves the issue, gather the following before contacting **[support@4minds.ai](mailto:support@4minds.ai)**:
* The exact prompt you sent
* Which Copilot host you're using (Teams / Office web / Office desktop)
* Your 4MINDS workspace name and email
* A screenshot of any error card the agent returned
# Microsoft Fabric
Source: https://docs.4minds.ai/microsoft-fabric
## Overview
The 4MINDS platform integrates with **Microsoft Fabric**, allowing you to connect your Fabric workspaces and import data directly into your 4MINDS datasets. You can browse and import files from **Fabric Lakehouses** and query tables from **Fabric Warehouses** — all from within the 4MINDS interface.
***
## Getting Started
### Prerequisites
* A Microsoft account with access to one or more Fabric workspaces
* Appropriate permissions on the Lakehouses or Warehouses you want to access
### Connecting Your Microsoft Fabric Account
1. Open **Integrations** from the main navigation bar in 4MINDS.
2. Find the **Microsoft Fabric** integration and click **Connect**.
3. A Microsoft sign-in window will open. Log in with the Microsoft account that has access to your Fabric workspaces.
4. Grant the requested permissions when prompted. The platform requires access to your Fabric workspaces and data.
5. Once authenticated, the integration will show as **Connected** along with your Microsoft email address.
> **Note:** The connection uses secure OAuth 2.0 authentication. Your Microsoft password is never stored by 4MINDS. Access tokens are encrypted and refreshed automatically.
### Disconnecting
To disconnect your Fabric account, open **Integrations** from the main navigation bar, find Microsoft Fabric, and click **Disconnect**. This removes your stored credentials and revokes access.
***
## Working with Lakehouses
A **Lakehouse** in Microsoft Fabric is a file-based data store. You can browse and import files stored in the Lakehouse's `Files` directory.
### Browsing Lakehouses
1. When creating or updating a dataset, select **Microsoft Fabric** as the data source.
2. Choose **Lakehouses** from the root view.
3. You will see a list of all Lakehouses across your Fabric workspaces, grouped by workspace name.
4. Select a Lakehouse to browse its files.
### Importing Files from a Lakehouse
1. After selecting a Lakehouse, the platform displays all files in its `Files` directory.
2. Use the **search bar** to filter files by name.
3. Select one or more files using the checkboxes.
4. Click **Add Files** to stage them for import into your dataset.
### Supported File Types
The following file formats can be imported from Lakehouses:
| Category | Formats |
| ------------- | ----------------------------------- |
| **Data** | CSV, JSON, JSONL, TXT, TSV, Parquet |
| **Documents** | PDF, DOCX, ODT, XLSX, HTML |
| **Images** | PNG, JPG, JPEG, GIF, BMP, TIFF |
| **Archives** | ZIP (extracted automatically) |
### File Size Limit
Individual files must be **100 MB or smaller**. Files exceeding this limit are filtered out automatically.
***
## Working with Warehouses
A **Warehouse** in Microsoft Fabric is a SQL-based data store. You can browse tables and views, and import their data into your 4MINDS datasets.
### Browsing Warehouses
1. When creating or updating a dataset, select **Microsoft Fabric** as the data source.
2. Choose **Warehouses** from the root view.
3. You will see a list of all Warehouses across your Fabric workspaces, grouped by workspace name.
4. Select a Warehouse to view its tables and views.
### Importing Data from a Warehouse
1. After selecting a Warehouse, you will see a list of all tables and views (excluding system tables).
2. Each entry shows the schema name and table name (e.g., `dbo.SalesData`).
3. Select one or more tables using the checkboxes.
4. Click **Add Files** to import the table data into your dataset.
> **Note:** Warehouse table data is imported as CSV. Each import retrieves up to **10,000 rows** from the selected table. If a table contains more than 10,000 rows, the import will include the first 10,000 rows.
***
## Dataset Sync
Once you have imported data from Fabric into a dataset, you can set up **automatic syncing** to keep your dataset up to date with changes in Fabric.
### How Sync Works
* **Lakehouse sync:** The platform periodically checks the connected Lakehouse for new, updated, or removed files and syncs them into your dataset.
* **Warehouse sync:** The platform re-queries the connected Warehouse tables and updates the dataset with the latest data.
### Sync Options
* **Sync specific items:** Sync only selected files or tables.
* **Sync all:** Automatically pick up new files or tables added to the Lakehouse or Warehouse.
***
## Navigation & UI Features
### Breadcrumb Navigation
When browsing Fabric resources, a breadcrumb trail at the top of the view shows your current location (e.g., **Fabric > Lakehouses > My Lakehouse > Files**). Click any breadcrumb segment to navigate back to that level.
### Multi-Select
Use checkboxes to select multiple files or tables at once for bulk import.
### Search
Use the search bar when browsing Lakehouse files to quickly filter by file name.
***
## Troubleshooting
| Issue | Solution |
| ---------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| **"Not connected" message** | Open Integrations from the main nav and connect your Microsoft Fabric account. |
| **No Lakehouses or Warehouses listed** | Verify that your Microsoft account has access to at least one Fabric workspace with Lakehouses or Warehouses. |
| **Files missing from Lakehouse** | Only supported file types (see table above) under 100 MB are shown. Check that your files meet these requirements. |
| **Access denied error** | Your Microsoft account may not have the required permissions on the Fabric workspace. Contact your Fabric administrator. |
| **Connection expired** | The platform automatically refreshes your connection. If issues persist, disconnect and reconnect your account from the Integrations tab. |
| **Warehouse query returns partial data** | Warehouse imports are limited to 10,000 rows per table. For larger tables, consider filtering the data in Fabric before importing. |
***
## Security & Privacy
* **OAuth 2.0:** Authentication uses industry-standard OAuth 2.0. Your Microsoft credentials are never stored by 4MINDS.
* **Encrypted tokens:** All access and refresh tokens are encrypted at rest.
* **Automatic token refresh:** Tokens are refreshed automatically before they expire, so you stay connected without re-authenticating.
* **Per-user access:** Each user connects their own Microsoft account. Your Fabric connection is not shared with other users on the platform.
* **Scoped permissions:** The integration only requests the permissions needed to browse and read data from your Fabric workspaces.
***
## FAQ
**Q: Which Fabric workspaces can I access?** A: You can access all workspaces where your Microsoft account has at least read permissions.
**Q: Can I import data from Fabric into an existing dataset?** A: Yes. You can add Fabric files and tables to both new and existing datasets, and even combine them with data from other sources.
**Q: How often does dataset sync run?** A: Sync frequency depends on your dataset's sync configuration. You can also trigger a manual sync at any time.
**Q: What happens to my data if I disconnect my Fabric account?** A: Previously imported data remains in your datasets. However, automatic syncing will stop, and you will not be able to import new data from Fabric until you reconnect.
**Q: Can I connect multiple Microsoft accounts?** A: Each user account on 4MINDS supports one Microsoft Fabric connection at a time. To switch accounts, disconnect the current one and connect with a different Microsoft account.
# Microsoft Foundry
Source: https://docs.4minds.ai/microsoft-foundry
## Table of Contents
* [Overview](#overview)
* [Azure Setup](#azure-setup)
* [Create an Azure AI Foundry Project](#create-an-azure-ai-foundry-project)
* [Deploy a Model](#deploy-a-model)
* [Find Your Credentials](#find-your-credentials)
* [Grant Azure RBAC Role (Required for OAuth)](#grant-azure-rbac-role-required-for-oauth)
* [OAuth App Registration (Admin)](#oauth-app-registration-admin)
* [Connecting to 4MINDS](#connecting-to-4MINDS)
* [Method 1: OAuth (Recommended)](#method-1-oauth-recommended)
* [Method 2: API Key](#method-2-api-key)
* [Creating an External Model](#creating-an-external-model-with-microsoft-foundry)
* [Guide Me (Easy Mode)](#guide-me-easy-mode)
* [Advanced Mode](#advanced-mode)
* [Using Foundry Models](#using-foundry-models)
* [Browsing Deployments](#browsing-deployments)
* [Chat Completions](#chat-completions)
* [Troubleshooting](#troubleshooting)
***
## Overview
The Microsoft Foundry integration connects your Azure AI Foundry project to the 4MINDS platform, allowing you to:
* **Browse deployments** from your Azure AI Foundry project
* **Register models** deployed in Foundry as 4MINDS models
* **Run chat completions** against Foundry-hosted models (GPT-4o, GPT-4, Llama, Mistral, etc.)
* **Fine-tune models** using 4MINDS datasets with Foundry compute
Two authentication methods are supported:
| Method | Best For | Requires Admin Consent? | Token Refresh |
| ----------- | ------------------------------------------------------------- | ----------------------- | -------------------------------- |
| **OAuth** | Organizations with Azure AD admin consent granted | Yes | Automatic |
| **API Key** | Individual users or orgs where admin consent is not available | No | Not required (keys don't expire) |
***
## Azure Setup
### Create an Azure AI Foundry Project
1. Go to [Azure AI Foundry](https://ai.azure.com) and sign in with your Microsoft account
2. Click **+ New project**
3. Select or create an **Azure AI hub** resource
4. Give your project a name and select a region
5. Click **Create project**
> **Tip:** Choose a region close to your users for lower latency. Models available vary by region.
### Deploy a Model
Before connecting to 4MINDS, deploy at least one model in your Foundry project:
1. In your Azure AI Foundry project, go to **Model catalog** in the left sidebar
2. Browse or search for a model (e.g., `gpt-4o`, `gpt-4`, `Mistral-large`)
3. Click **Deploy**
4. Choose a deployment name and configure settings:
* **Deployment name**: A unique identifier (e.g., `gpt-4o`)
* **Model version**: Select the desired version
* **Deployment type**: Standard or Provisioned
* **Rate limits**: Configure tokens-per-minute as needed
5. Click **Deploy**
### Find Your Credentials
Both the endpoint URL and API key live in the same place. The steps below show you how to find each of them.
#### Option A: From Azure AI Foundry (ai.azure.com) — Easiest
1. Go to [Azure AI Foundry](https://ai.azure.com) and sign in
2. Click on your project/resource
3. On the **Overview** page you'll see:
* **Microsoft Foundry project endpoint** — copy this URL
* **API Key** — click the copy icon to copy it
4. The endpoint URL looks like:
```text theme={null}
https://your-project-name.region.inference.ml.azure.com
```
#### Option B: From Azure Portal (portal.azure.com)
1. Open your project in [Azure AI Foundry](https://ai.azure.com)
2. In the left sidebar, go to **Resource Management > Keys and Endpoint**
3. Copy the **Endpoint** URL
4. Under **Keys**, you'll see **Key 1** and **Key 2** — click the copy icon next to either key
> **Important:** Copy the full endpoint URL including `https://`. Do not include a trailing slash — 4MINDS will strip it, but it's cleaner to paste it correctly.
> **Security:** Treat your API key like a password. Do not share it or commit it to source control. You can regenerate keys at any time — regenerating a key immediately invalidates the old one.
> **Tip:** Azure provides two keys so you can rotate them without downtime. Use Key 1 for your connection, and if you need to rotate, switch to Key 2 before regenerating Key 1.
### Grant Azure RBAC Role (Required for OAuth)
When connecting via **OAuth**, your Azure user account needs an RBAC role on the Foundry resource to list deployments and invoke models. The API Key method bypasses this because the key itself carries the permissions.
1. Open your Foundry resource in [Azure Portal](https://portal.azure.com)
2. In the left sidebar, click **Access control (IAM)**
3. Click **+ Add > Add role assignment**
4. Assign one of the following roles to your user account (or an Azure AD group you belong to):
* **Azure AI Developer** — recommended; grants access to Foundry data plane APIs
* **Cognitive Services Contributor** — broader alternative
5. Click **Review + assign**
> **Note:** Azure RBAC changes can take up to a minute to propagate. If your first connection attempt fails with a permissions error, wait 30–60 seconds and try again. See [Troubleshooting](#permissions-still-propagating-after-oauth-connect).
### OAuth App Registration (Admin)
If your organization wants to use the OAuth connection method, an Azure AD administrator must register and consent to the 4MINDS application:
1. Navigate to **Azure Portal > Microsoft Entra ID > App registrations**
2. Find the 4MINDS Foundry app registration (or create one if self-hosting)
3. Go to **API permissions**
4. Ensure the following permissions are configured:
* `https://ai.azure.com/.default` — Access Azure AI services on behalf of the signed-in user (covers all delegated permissions needed by Foundry)
* `offline_access` — Issue a refresh token so 4MINDS can refresh the access token without re-prompting
5. Click **Grant admin consent for \[Your Organization]**
6. Confirm the consent prompt
> **Note:** If you see "Admin approval required" or "AADSTS65001" when trying to connect via OAuth, your Azure AD admin has not yet granted consent. Use the [API Key method](#method-2-api-key) as an alternative.
***
## Connecting to 4MINDS
### Method 1: OAuth (Recommended)
OAuth signs you in with your Microsoft account and automatically refreshes access tokens 5 minutes before they expire, so long-running sessions don't break. Connecting is a two-step process: authorize Microsoft, then provide the Foundry endpoint URL.
**Steps:**
1. Open **Integrations** from the main navigation in 4MINDS and select **Microsoft Foundry** (or do this during onboarding)
2. Make sure the **OAuth** tab is selected (it is selected by default)
3. Click **Connect with Microsoft**
4. A Microsoft login popup will open — sign in with your Azure account and grant the requested permissions
* If the popup is blocked, enable popups for the site and try again
5. After successful authentication, the popup closes automatically and 4MINDS shows an "OAuth connection successful" message
6. **Enter your Endpoint URL** — see [Find Your Credentials](#find-your-credentials) for where to find it:
```text theme={null}
https://your-project-name.region.inference.ml.azure.com
```
7. Click **Save Endpoint URL**
Your Foundry deployments will load automatically once the endpoint URL is saved. The endpoint URL is preserved if you later disconnect and reconnect via OAuth.
> **Note:** The OAuth popup may close before the backend finishes exchanging the authorization code. 4MINDS polls the connection status for several seconds afterward, so give it a moment if the success state doesn't appear instantly.
### Method 2: API Key
Use the API Key method when:
* Your organization's Azure AD admin has not granted consent for OAuth
* You prefer not to use OAuth
* You're connecting from a personal Azure account without organizational admin access
Unlike OAuth, the API Key method takes the endpoint URL and key together in a single step and stores them as an `api_key` connection.
**Steps:**
1. Open **Integrations** from the main navigation in 4MINDS and select **Microsoft Foundry** (or do this during onboarding)
2. Click the **API Key** tab
3. Enter your **Endpoint URL** — see [Find Your Credentials](#find-your-credentials) for where to find it:
```text theme={null}
https://your-project-name.region.inference.ml.azure.com
```
4. Enter your **API Key** — same place as the Endpoint URL, see [Find Your Credentials](#find-your-credentials)
5. Click **Connect**
6. 4MINDS tests the connection against the Foundry endpoint and saves your credentials if successful
> **Note:** API keys do not expire automatically, but they can be regenerated by anyone with access to the Azure AI Foundry project settings. If your connection stops working with a 401 error, check that your key hasn't been regenerated — and reconnect with the new key.
> **Tip:** The API Key method skips the Azure RBAC role requirement because the key itself carries the permissions needed to call Foundry endpoints.
***
## Creating an External Model with Microsoft Foundry
There are two ways to create a model backed by a Microsoft Foundry deployment: **Guide Me** (easy mode) and **Advanced** mode.
### Guide Me (Easy Mode)
1. From the home screen, click **Guide Me**
2. On the **Model Source** step, you'll see two options:
* "Build custom AI"
* A list of external model providers
3. Select **Microsoft Foundry** from the list of providers
4. Connect to your Foundry account using either [OAuth](#method-1-oauth-recommended) or [API Key](#method-2-api-key)
5. Once connected, you'll see a list of your deployed models — select the one you want to use
6. Click **Next** to proceed to the **Upload Data** step
### Advanced Mode
1. Navigate to the **Models** page
2. Click the **Create Model** button
3. Enter a **model name**
4. Under **Or connect an external model**, select **Microsoft Foundry**
5. Connect to your Foundry account if not already connected
6. Select a deployed model from the list
7. Click **Next** to proceed to the **Upload Data** step
***
## Using Foundry Models
### Browsing Deployments
Once connected, your Foundry deployments appear automatically:
* Each deployment shows its **name**, **model type**, and **status**
* Click on a deployment to select it
### Chat Completions
Registered Foundry models can be used anywhere in 4MINDS that supports model selection:
* Chat conversations
* Model comparison
* Fine-tuning evaluation
Chat completions are routed through the Azure OpenAI-compatible endpoint (`/openai/deployments/{deployment}/chat/completions`) on your Foundry project, with streaming supported via server-sent events.
***
## Troubleshooting
### "Admin approval required" when connecting via OAuth
**Cause:** Your Azure AD tenant requires admin consent for the 4MINDS application.
**Solutions:**
1. Ask your Azure AD administrator to [grant admin consent](#oauth-app-registration-admin)
2. Use the [API Key connection method](#method-2-api-key) instead — no admin consent required
### "Invalid credentials" or "Connection test failed" (API Key)
**Cause:** The endpoint URL or API key is incorrect.
**Solutions:**
1. Verify the endpoint URL is copied exactly from Azure AI Foundry (including `https://`, no trailing slash)
2. Re-copy the API key from Azure AI Foundry > **Overview** or **Keys and Endpoint**
3. Check that the API key hasn't been regenerated since you last copied it
4. If your key was rotated, reconnect with the new key
### "No deployments found"
**Cause:** No models are deployed in the Foundry project, or the endpoint URL points to the wrong project.
**Solutions:**
1. Verify you have at least one [deployed model](#deploy-a-model) in your Azure AI Foundry project
2. Check that the endpoint URL matches the correct project
3. Ensure the deployed model's status is "Succeeded" in Azure AI Foundry
### Permissions still propagating after OAuth connect
**Cause:** Azure RBAC role assignments can take up to a minute to propagate after you grant them.
**Solutions:**
1. Wait 30–60 seconds and retry — 4MINDS automatically retries deployment listing once on auth failures, but if the delay is longer you may need to refresh the page
2. Confirm you have the **Azure AI Developer** or **Cognitive Services Contributor** role on the Foundry resource (see [Grant Azure RBAC Role](#grant-azure-rbac-role-required-for-oauth))
3. If the error persists past a minute, try disconnecting and reconnecting
### OAuth token refresh failed
**Cause:** The OAuth refresh token has expired or been revoked. Refresh tokens are long-lived but can be invalidated by password changes, conditional access policy updates, or admin revocation.
**Solutions:**
1. Disconnect and reconnect via OAuth
2. Check that admin consent is still granted in Azure AD
3. Switch to API Key method if the issue persists
> **Note:** This error only applies to OAuth connections. API Key connections do not use refresh tokens and are unaffected.
### Connection works but chat completions fail
**Cause:** The selected deployment may have rate limits, be in a failed state, or the model may not support the requested operation.
**Solutions:**
1. Check the deployment status in Azure AI Foundry — ensure it shows "Succeeded"
2. Verify rate limits haven't been exceeded (check Azure portal for 429 errors)
3. Ensure the deployment supports chat completions (some models only support embeddings or completions)
4. Try a different deployment to isolate the issue
### OAuth popup closes but 4MINDS shows "Authentication cancelled or failed"
**Cause:** The popup closed before the backend finished exchanging the authorization code with Microsoft, or the popup was blocked.
**Solutions:**
1. Enable popups for the 4MINDS site in your browser settings
2. On Safari: **Safari > Settings > Websites > Pop-up Windows** and allow popups for 4MINDS
3. Wait a few seconds — 4MINDS polls for connection status after the popup closes and may still recover
4. If it persists, click **Connect with Microsoft** again
# Explore the Model Dashboard
Source: https://docs.4minds.ai/models
A model is a machine learning system — typically a large language model (LLM) — that's been trained on data to recognize patterns and generate responses. In the 4MINDS platform, you can create personalized models adapted to your organization's specific data and use cases. The Model Dashboard displays all your models with their current training status. View key performance indicators at a glance: number of models ready for deployment, token processing speed, average response time, and success rate across requests. Click any model to view detailed metrics and configuration settings.
![Screenshot 2026 06 01 At 11 34 48 AM]( "Screenshot 2026 06 01 At 11 34 48 AM")
![Screenshot 2026 06 01 At 11 34 48 AM]( "Screenshot 2026 06 01 At 11 34 48 AM")
### View Options
Toggle between two display modes to suit your preference:
* **Card view** - Visual grid layout showing models as cards
* **Table view** - Compact tabular format for detailed information
### Base model
When deploying directly on 4MINDS, your custom model runs on `gpt-oss-120b` — no in-platform base-model picker.
**Direct base-model selection has been deprecated.** To use other foundation models, connect an external provider — see [Creating a Model in Advanced Mode](#creating-a-model-in-advanced-mode) below for details.
### Creating a Model in Advanced Mode
1. **Step 1 of 3 — Model Settings.** Click the ‘**Create Model**’ button and select ‘**Advanced Mode**’ to begin creating your model. Configure the basics for your model, then click '**Next**':
* **Name** *(required)* — Enter a name for your model.
* **Description** *(required)* — Briefly describe what the model is for.
* **Use Case** *(required)* — Select the use case that best matches your model's purpose.
* **Persona** *(optional)* — Apply a persona to shape the model's tone. See [Personas](/persona-configuration).
* **Category** *(optional)* — Tag the model with a category for organization.
* **Base model / External provider** — By default, the model runs on **`gpt-oss-120b`**. To use a different foundation model, select an externally connected provider (e.g. [Amazon Bedrock](/bedrock), Google Vertex AI, [Amazon SageMaker](/integrations#amazon-sagemaker), or [Microsoft Foundry](/microsoft-foundry)) — see [Add Integrations & Data Sources](/integrations) for setup.
![Screen Shot2025 11 01at7 40 17PM Pn]( "Screen Shot2025 11 01at7 40 17PM Pn")
![Screen Shot2025 11 01at7 40 17PM Pn]( "Screen Shot2025 11 01at7 40 17PM Pn")
###
2. **Step 2 of 3 — Upload data.** Add training data to your model. Select a data source using the tab selector at the top, or pick a pre-existing dataset from the **Or use an existing dataset** dropdown at the bottom.
Available data source tabs:
* **Upload** *(default)* — Select local files using the **Choose a File** button. Multiple files can be uploaded at once. Supported formats include:
* Data: CSV, TSV, PARQUET, JSON, JSONL
* Documents: PDF, DOCX, MD, TXT
* Code: PY, JS, TS, SQL, and many others
* Archives: ZIP
Maximum file size is **2,000 MB**, with a total upload cap of **2,000 MB**.
* **Integrations** — Pull data from a [connected integration](/integrations) (e.g. Hugging Face, S3, Google Drive).
* **URL** — Import data from a public web URL.
Click '**Add Files**' to attach the selected files to the model.
![Screen Shot2025 11 01at7 48 02PM Pn]( "Screen Shot2025 11 01at7 48 02PM Pn")
![Screen Shot2025 11 01at7 48 02PM Pn]( "Screen Shot2025 11 01at7 48 02PM Pn")
* Click '**Next**' to proceed to the next step.
3. **Step 3 of 3 — Review.** Verify your configuration before model creation begins. The review screen shows a read-only summary of the choices made in the previous steps in a two-column label/value layout:
| Field | Description |
| --------------- | ------------------------------------------------------------- |
| **Name** | The model's display name (e.g. "Forecasting for Oil and Gas") |
| **Description** | Brief summary of the model's purpose |
| **Base Model** | The underlying foundation model (e.g. `GPT-OSS-120B`) |
| **Use Case** | The intended application domain (e.g. "Finance") |
| **Persona** | The assigned persona, or `None` if not configured |
| **Category** | The classification category for the model (e.g. "Finance") |
| **Dataset** | The training dataset selected in Step 2 |
Review each field for accuracy. To correct a value, navigate back to the relevant step. When everything looks correct, click ‘**Create Model**‘ to finalize your model creation.
# Stay Updated with Real-Time Notifications
Source: https://docs.4minds.ai/notifications
Receive instant alerts about dataset processing, model readiness, import completion, and system errors. Stay informed across all your devices with real-time WebSocket updates.
## Overview
The 4MINDS platform keeps you informed with real-time notifications about critical events in your workflow. Whether you're processing datasets, training models, or importing data from external sources, you'll receive instant updates so you can act quickly.
## Viewing Real-Time Notifications
To view real-time notifications, click the bell icon in the top-right corner of the screen, next to the Cloud Shell icon.
![Screen Shot2025 11 06at7 52 10PM Pn]()
![Screen Shot2025 11 06at7 52 10PM Pn]()
## Notification types
### Dataset processing complete
Get notified when your uploaded datasets finish processing through the ETL and Graph engines. This alert confirms your data is ready to be used for model training or testing.
### Model ready for use
Receive an alert when your model completes training and is deployed. You can immediately start testing or integrating the model into your applications.
### Import completion
When importing data from external sources like HuggingFace or Databricks, you'll receive confirmation once the import finishes successfully.
### Error alerts
If something goes wrong—failed uploads, processing errors, or connection issues—you'll get immediate notification with details about what happened and suggested next steps.
## How notifications work
### Real-time delivery
Notifications use WebSocket connections to deliver updates instantly. You don't need to refresh your browser or check back manually—alerts appear as soon as events occur.
### Cross-device sync
Notifications sync across all devices where you're logged in. Start a dataset upload on your laptop, and get the completion alert on your phone.
## Managing notifications
You can view all notifications in your Control Center. Recent alerts appear in the notification panel, where you can:
* Review notification history
* Mark notifications as read
* Jump directly to the relevant resource (dataset, model, or import)
* Clear old notifications to keep your workspace organized
Notifications help you stay productive by eliminating the need to constantly monitor long-running processes. Focus on other work, and let 4MINDS alert you when action is needed.
# Personas
Source: https://docs.4minds.ai/persona-configuration
Create and manage AI personality templates that define tone, communication style, and behavioral traits for different use cases.
## Overview
Personas let you create different AI personality templates for various use cases. Each persona defines the tone, communication style, and key behavioral traits that shape how the AI interacts with users - whether that's a technical analyst, customer service representative, or creative writer.
**Personas define tone and personality — not role context.** Use personas to shape how your model communicates (professional, technical, casual, etc.). If you need to define a system-level role or context (e.g. "You are a cybersecurity expert in a SOC environment"), configure that in a SYMI sub-agent task definition instead.
## Accessing personas
Select **Personas** from the top navigation bar to access the Personas management dashboard.
### Dashboard features
* **Search bar**: Find personas by name.
* **Tone filter**: Filter by tone using the **All Tones** dropdown. Available values: All Tones, Professional, Friendly, Casual, Formal, Technical, Creative.
* **Start**: Click **Start** next to any persona to activate it in your chat session.
* **+ Create Persona**: Create a new custom persona.
***
## Persona templates
Choose from six pre-configured templates to get started quickly:
### Professional Assistant
| Attribute | Value |
| --------------- | ------------------------------------------------------------------- |
| **Description** | A polished, business-oriented persona for enterprise communications |
| **Tone** | Professional |
| **Key Traits** | Analytical, Precise, Detailed |
| **Best For** | Corporate environments, formal business interactions |
### Friendly Support Agent
| Attribute | Value |
| --------------- | ------------------------------------------------- |
| **Description** | A warm, approachable persona for customer service |
| **Tone** | Friendly |
| **Key Traits** | Empathetic, Helpful, Patient |
| **Best For** | Customer support, help desks, user assistance |
### Technical Expert
| Attribute | Value |
| --------------- | -------------------------------------------------------------- |
| **Description** | A knowledgeable, detail-oriented persona for technical support |
| **Tone** | Technical |
| **Key Traits** | Knowledgeable, Innovative, Detailed |
| **Best For** | IT support, developer assistance, technical documentation |
### Creative Collaborator
| Attribute | Value |
| --------------- | ----------------------------------------------------------------- |
| **Description** | An innovative, imaginative persona for brainstorming and ideation |
| **Tone** | Creative |
| **Key Traits** | Adaptable, Creative, Innovative |
| **Best For** | Brainstorming sessions, content creation, ideation workshops |
### Formal Consultant
| Attribute | Value |
| --------------- | ---------------------------------------------------------------------------- |
| **Description** | A sophisticated, authoritative persona for high-stakes business interactions |
| **Tone** | Formal |
| **Key Traits** | Knowledgeable, Precise, Reliable |
| **Best For** | Executive communications, consulting, high-stakes negotiations |
### Casual Mentor
| Attribute | Value |
| --------------- | ------------------------------------------------------------------- |
| **Description** | A relaxed, down-to-earth persona for informal guidance and learning |
| **Tone** | Casual |
| **Key Traits** | Helpful, Supportive, Encouraging |
| **Best For** | Training, onboarding, informal learning environments |
***
## Creating a custom persona
If the templates don't meet your needs, click **+ Create Persona** in the top right to open the creation panel. The process has **3 steps**.
### Step 1: Basic information
To create a persona, navigate to the ‘**Personas**‘ tab and click ‘**Create Persona**‘.
Fill in the following fields and click ‘**Next**‘:
* **Persona Name** *(required)* — Enter a name for your persona.
* **Description** — Briefly describe the persona's purpose and communication style.
![Screen Shot2025 10 09at10 47 47AM Pn]( "Screen Shot2025 10 09at10 47 47AM Pn")
![Screen Shot2025 10 09at10 47 47AM Pn]( "Screen Shot2025 10 09at10 47 47AM Pn")
### Step 2: Tone and key traits
Select the tone of voice from the dropdown and key traits to define this persona's behavior.
![Screen Shot2025 10 09at10 51 33AM Pn]( "Screen Shot2025 10 09at10 51 33AM Pn")
![Screen Shot2025 10 09at10 51 33AM Pn]( "Screen Shot2025 10 09at10 51 33AM Pn")
### Step 3: Review and Create
Review your persona configuration before creating. Click ‘**Create Persona**‘.
![Screenshot 2026 05 28 At 6 53 38 PM]( "Screenshot 2026 05 28 At 6 53 38 PM")
![Screenshot 2026 05 28 At 6 53 38 PM]( "Screenshot 2026 05 28 At 6 53 38 PM")
Your persona has been successfully created and is now ready to interact.
![Screen Shot2025 10 09at11 01 13AM Pn]()
![Screen Shot2025 10 09at11 01 13AM Pn]()
***
## Persona status
Each persona can have one of the following statuses:
| Status | Description |
| ---------- | ------------------------------------------------- |
| **Draft** | Persona is being configured and is not yet active |
| **Active** | Persona is live and available for use |
***
## Managing personas
The persona table displays the following information:
| Column | Description |
| -------------- | -------------------------------------- |
| **Persona** | Name of the persona |
| **Tone** | Communication style |
| **Key Traits** | Behavioral characteristics |
| **# Models** | Number of AI models using this persona |
| **Created** | Creation date |
| **Status** | Current status (Draft/Active) |
| **Actions** | Edit, duplicate, or delete options |
***
## Next steps
Explore detailed definitions of customer features and industry concepts
# Platform Architecture
Source: https://docs.4minds.ai/platform-architecture
How 4MINDS takes a structural approach to building trustworthy, scalable AI systems that differ fundamentally from model-centric platforms.
## A Different Starting Point
Most AI systems assume the model is the intelligence. 4MINDS starts from a different assumption: **intelligence emerges from structure, not just scale.**
Language models are powerful components, but not decision makers, memory stores, or execution engines. Instead, intelligence is distributed across a structured system designed to make reasoning explicit and behavior governable.
## How 4MINDS Differs from Other Platforms
Most AI platforms fall into three categories:
| Category | Approach | Limitation |
| ----------------------- | ---------------------------------------------------- | ------------------------------------------------- |
| **Model-centric** | Access to LLMs with prompt templates and fine-tuning | Intelligence assumed to emerge from scale alone |
| **RAG systems** | Vector databases attached to models | Single-hop retrieval, struggles with verification |
| **Tool-enabled agents** | Models call external tools directly | Risks around safety, auditability, and control |
4MINDS doesn't fit these categories. It treats AI as a cognitive platform built around explicit reasoning, verifiable knowledge, governed execution, and controlled adaptation.
## Constellation: The Cognitive Reasoning System
Constellation is the structured cognitive system at the core of 4MINDS. It performs analysis, verification, contradiction handling, confidence calibration, and planning—but critically, **it is not an autonomous actor**.
Constellation doesn't execute actions, hold credentials, mutate system state, or update model parameters. It reasons about the world; it doesn't act upon it.
### How Constellation Works
Within 4MINDS, Constellation operates between draft generation and final synthesis:
1. The Response Engine generates an initial draft using structured retrieval and long-term memory
2. The draft, evidence, and context pass to Constellation
3. Constellation performs multi-stage reasoning over the draft
4. Structured outputs influence final synthesis, memory updates, and capability invocation
5. The Response Engine produces the final response under explicit constraints
No response reaches users without passing through verification and confidence calibration.
### Structured Multi-Hop Reasoning
Constellation implements reasoning as a directed acyclic graph (DAG) of specialized agents, each with narrowly defined responsibilities:
* **Claim decomposition**: Breaking down assertions into verifiable components
* **Evidence verification**: Evaluating claims against knowledge graphs, documents, and memory
* **Contradiction detection**: Surfacing conflicts explicitly rather than hiding them
* **Confidence assessment**: Computing confidence based on evidence coverage and reasoning completeness
* **Synthesis constraints**: Guiding final response generation
Agents execute in parallel where possible and in sequence where dependencies require, ensuring predictable performance without unbounded cognition.
### Confidence as a Computed Signal
In Constellation, confidence isn't stylistic—it's computed. Confidence reflects evidence coverage, contradiction severity, memory alignment, and reasoning completeness. It directly influences how assertive responses may be and whether uncertainty must be stated explicitly.
### MoE-Aware Cognition
Constellation explicitly supports Mixture-of-Experts (MoE) architectures. In MoE systems, outputs may originate from multiple expert subspaces with different priors. Constellation treats expert output as evidence, not authority—detecting cross-expert contradictions, calibrating confidence accordingly, and constraining synthesis to maintain coherence.
### Safety by Design
Because Constellation never executes actions, never mutates state, and never adapts weights implicitly, it's structurally resistant to prompt injection, privilege escalation, and silent behavior drift. Every decision is traceable, every output inspectable, and every downstream effect governed by explicit policy.
## Synthesis Graph™: Structured Semantic Substrate
Flat vector search struggles with scale and semantic dilution. Traditional graph databases aren't designed for continuous ingestion or real-time inference. The 4MINDS Synthesis Graph™ is a hierarchical semantic substrate that supports large-scale knowledge representation, fast retrieval, and verifiable reasoning.
**The Synthesis Graph™ is not directly accessible via API.** There is no graph query endpoint — the graph is queried internally during model inference. To retrieve knowledge from the graph, send a request to a model inference endpoint; the model traverses the graph as part of generating its response.
### Hierarchical Structure
The Synthesis Graph™ is organized as a **parent graph** composed of many **sub-graphs**:
* **Parent graph**: Represents broad semantic regions of knowledge
* **Sub-graphs**: Coherent clusters of related information, intentionally bounded for consistency
* **Centroids**: Semantic anchors summarizing each sub-graph's meaning
* **Super centroids**: Higher-level anchors representing collections of sub-graphs
This structure scales to millions of nodes without collapsing into noise. Growth is absorbed by creating new sub-graphs rather than expanding a flat structure indefinitely.
### Routed and Parallel Search
Queries are first evaluated against high-level centroids to identify relevant semantic regions. Multiple regions can be selected simultaneously for ambiguous or multi-topic queries. Detailed retrieval occurs only within those bounded regions—avoiding exhaustive global search while preserving recall and relevance.
Result: fast, predictable retrieval even as the graph grows very large.
### Evidence and Contradiction Preservation
The Synthesis Graph™ preserves evidence and disagreement rather than eliminating it. When sources conflict, those conflicts are represented structurally. This allows Constellation to detect contradictions explicitly and adjust confidence accordingly. The graph doesn't resolve truth—it provides structured context for verification.
### Role in the Cognitive Pipeline
The Synthesis Graph™ supports:
* **Response Engine**: High-quality contextual grounding
* **Constellation**: Verification and contradiction detection
* **Memory System**: Aligning new information against existing knowledge
* **Capability planning**: Grounding actions in structured understanding
The graph itself doesn't reason, learn implicitly, or execute actions.
### Knowledge Evolution
Knowledge is continuously added through governed ingestion. As the graph evolves, new sub-graphs are created when needed, existing sub-graphs are refined to preserve coherence, and historical context is retained rather than overwritten. The system adapts to change without disruptive rebuilds.
## Memory Without Drift
Many AI systems "learn" by adjusting weights or storing unstructured conversation history, leading to drift and inconsistency. 4MINDS separates memory from behavior:
* Long-term memory is explicit, typed, and confidence-scored
* Memory persists across sessions but doesn't silently change model behavior
* Memory informs reasoning without overriding it
## Model-Agnostic by Design
4MINDS supports dense models and Mixture-of-Experts (MoE) architectures without changing the cognitive pipeline. In MoE systems, expert disagreement is expected—4MINDS treats expert output as evidence to verify rather than authority to trust.
## Controlled Adaptation with Ghost Weights™
AI systems need to adapt over time, but traditional approaches introduce significant trade-offs. 4MINDS uses Ghost Weights™ as a governed alternative.
### Traditional Adaptation Approaches
| Approach | Best For | Limitations |
| -------------------------- | ------------------------------------------------------- | ----------------------------------------------------------------------------------- |
| **Reinforcement Learning** | Environments with clear rewards and reversible failures | Opaque credit assignment, irreversible updates, unpredictable drift |
| **Fine-tuning** | Static domain specialization with infrequent updates | Global parameter changes, difficult to isolate or reverse, accumulating brittleness |
Both approaches assume adaptation must occur through weight changes—problematic when behavior must be explainable, reversible, or tenant-specific.
### How Ghost Weights™ Work
Ghost Weights™ are bounded, reversible parameter overlays that modulate model behavior without altering the base model:
* **Bounded scope**: Changes remain localized to a small portion of parameter space
* **Atomic application**: Safe swap-in and swap-out without retraining
* **Tenant isolation**: No cross-contamination between deployments
* **Explicit governance**: Requires approval before activation
Ghost Weights™ are applied deliberately when adaptation is justified—not automatically in response to rewards or execution outcomes. The base model remains unchanged, preserving original capabilities.
### Adaptation Without Drift
Because Ghost Weights™ are explicitly applied, independently versioned, and fully reversible, they avoid the gradual compounding changes common in RL or repeated fine-tuning. System behavior stays predictable over long deployment lifetimes.
### Relationship to Reasoning and Memory
Within 4MINDS, adaptation isn't limited to model parameters:
* **Reasoning**: Handled through structured, multi-hop analysis
* **Learning**: Occurs through explicit long-term memory (typed, confidence-scored, governed)
* **Adaptation**: Ghost Weights™ reserved for cases where behavior itself must change
Most system evolution happens through memory and reasoning—not weight changes. This separation ensures AI systems can adapt responsibly while remaining auditable and controllable.
## Execution Without Losing Control
4MINDS introduces strict separation between intent and execution:
* Actions are planned declaratively
* Execution runs through a controlled runtime
* A security kernel enforces scope, rate limits, approvals, and kill switches
Even when autonomy is introduced, it's explicit, earned, and revocable.
## Built for Trust
Because reasoning, knowledge, memory, execution, and adaptation are all explicit and inspectable, every response traces back to:
* Evidence sources
* Reasoning steps
* Confidence assessments
* Governance decisions
This transparency is rarely achievable in model-centric platforms.
## Designed for Change
AI innovation moves fast - new architectures, training techniques, and paradigms emerge continuously. Many platforms tightly coupled to specific models or methods face repeated rewrites as technology shifts.
4MINDS treats this volatility as a design constraint. The platform doesn't bet on any single model, training technique, or paradigm. Instead, it's built so that reasoning, knowledge, memory, execution, and adaptation are decoupled - each layer evolves independently as the AI landscape changes.
### Cognitive Growth Without Drift
Future cognitive improvements come through richer reasoning stages, improved contradiction classification, and more nuanced uncertainty handling - not deeper recursion or uncontrolled autonomy. Because reasoning is explicit and bounded, these improvements don't destabilize the system.
### Knowledge That Scales With Change
The hierarchical Synthesis Graph™ doesn't encode fixed worldviews. It preserves evidence, relationships, and disagreement. When knowledge changes, the system absorbs new information locally rather than requiring wholesale retraining.
### Models as Replaceable Layers
Dense models, MoE systems, and future hybrid approaches integrate without altering how reasoning, memory, or governance operate. Model progress becomes an upgrade path, not a rewrite.
### Security That Scales With Capability
As AI systems gain new abilities, security risks grow in parallel. Because reasoning never executes directly and execution is always mediated through centralized governance, new capabilities are absorbed into existing control structures. Security doesn't erode as intelligence grows.
### Federated Intelligence Without Centralization
Because 4MINDS separates memory, reasoning, and adaptation, federated approaches can share insights as structure rather than raw data - enabling collective intelligence without sacrificing privacy or control.
## Beyond Reinforcement Learning
For over a decade, reinforcement learning (RL) has been positioned as the primary mechanism for improving AI systems. While effective in constrained environments, RL introduces opacity, instability, and governance challenges incompatible with enterprise and regulated deployments.
4MINDS is a post-reinforcement learning cognitive system that replaces reward-driven behavior with explicit reasoning, verification, structured memory, and governed execution.
### Why RL Falls Short for Enterprise AI
| RL Limitation | Impact |
| ----------------------------- | ------------------------------------------------------------------------------ |
| **Implicit learning** | Can't answer "why did the system do this?" or "which assumption was wrong?" |
| **Credit assignment failure** | Struggles to identify which decision caused an outcome in multi-step workflows |
| **Governance gaps** | Policy updates can't be easily audited or constrained at fine granularity |
| **Behavioral drift** | Silent changes to system behavior over time |
### How 4MINDS Replaces RL Functions
Rather than optimizing policies through trial and error, 4MINDS improves outcomes through deterministic cognition and auditable decision pathways:
| RL Function | 4MINDS Replacement |
| ------------------- | -------------------------------------- |
| Policy optimization | Explicit reasoning DAG (Constellation) |
| Reward signals | Verification + confidence scoring |
| Credit assignment | Claim-level tracing |
| Behavioral learning | Structured memory |
| Model adaptation | Ghost Weights™ |
| Safety constraints | Governance kernel |
### Operational Advantages
* **Determinism**: Identical inputs produce identical reasoning paths
* **Explainability**: Every decision is reconstructible
* **Stability**: Models don't drift unpredictably
* **Compliance**: Execution is auditable and governable
* **Scalability**: Works across domains, models, and tools
This approach surpasses RL-based systems in correctness, explainability, operational safety, and long-term reliability—while remaining model-agnostic and compatible with both dense and MoE architectures.
## Custom AI vs Democratized AI
As enterprises adopt AI at scale, two architectural philosophies have emerged:
| Approach | Philosophy | Best For |
| ------------------- | ------------------------------------------------------------------------------ | ---------------------------------------------------------------------------- |
| **Custom AI** | Intelligence tightly coupled to organization's data, workflows, and governance | High-risk operations, mission-critical processes, complex data relationships |
| **Democratized AI** | Advanced capabilities broadly accessible through standardized platforms | Knowledge work, cross-team collaboration, rapid adoption |
### The Custom AI Approach
Custom AI platforms (like Palantir) treat enterprise intelligence as inherently bespoke. Data is modeled explicitly, workflows are engineered with domain experts, and decision logic embeds directly into operational systems. This excels where data relationships are complex, processes are mission-critical, and central oversight is required.
**Trade-offs**: Requires significant upfront modeling, depends on specialized expertise, and can be slower to adapt to new use cases. Highly customized systems may struggle to keep pace with rapid AI evolution without continuous engineering investment.
### The 4MINDS Approach: Democratized by Design
4MINDS was built on a different premise: **intelligence should be accessible, adaptive, and governable without hand-engineering every use case.**
Rather than embedding intelligence into fixed workflows, 4MINDS provides a cognitive platform applicable across domains with minimal customization:
* Structured reasoning that adapts to new contexts
* Hierarchical knowledge graph organizing information dynamically
* Long-term memory accumulating understanding over time
* Governance layers applying consistently across use cases
### Intelligence as a System vs Intelligence as a Project
| Custom AI | 4MINDS |
| ------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------- |
| Intelligence delivered as a **project**: scoped, implemented, maintained for specific objectives | Intelligence delivered as a **system**: continuously operating, incrementally improving, reusable across contexts |
| Governance encoded into bespoke workflows | Governance as a platform-level construct |
| Intelligence concentrated within specific teams | Intelligence distributed across roles with consistent reasoning |
### Complementary, Not Mutually Exclusive
These approaches aren't mutually exclusive. Organizations may benefit from custom AI in core, high-risk operations alongside democratized AI for broader knowledge work. 4MINDS is designed to coexist with existing enterprise systems rather than replace them.
## Long-Term Implications
These architectural choices optimize for longevity, not demos. As models change and automation increases, platforms relying on implicit behavior struggle to maintain trust. Platforms embedding structure, governance, and verification scale without losing control.
The systems that endure won't be those that best exploit today's models, but those that survive tomorrow's disruptions. By separating cognition from models, structure from execution, and adaptation from learning, 4MINDS absorbs AI shifts rather than being displaced by them.
# Quick Start Guide
Source: https://docs.4minds.ai/quickstart
Get started with 4MINDS by building and testing your first AI model in minutes. This guide walks you through the essential steps.
## Create a 4MINDS account
Go to the [4MINDS website](https://app.4minds.ai/) and sign up for a free account.
![4MINDS signup page]( "4MINDS signup page")
You can sign up quickly using Single Sign-On with your Google, Microsoft, GitHub, AWS, Ping Identity, Okta, or LDAP account.
To view all available Enterprise SSO sign-in options, click **Sign in with Enterprise SSO** on the sign-in page.
![Screenshot 2026 06 09 At 12 11 24 PM]( "Screenshot 2026 06 09 At 12 11 24 PM")
## Create your first model
After logging in, navigate to the **Models** tab.
![Models tab]( "Models tab")
![Models tab]( "Models tab")
Click the '**Create Model**' button dropdown and select '**Easy Mode**' to begin setting up your model.
You'll be redirected to the **Create your AI model** screen. Click the '**Guide me**' text to continue.
![Screenshot 2026 05 28 At 1 47 59 PM]( "Screenshot 2026 05 28 At 1 47 59 PM")
![Screenshot 2026 05 28 At 1 47 59 PM]( "Screenshot 2026 05 28 At 1 47 59 PM")
## Easy Mode Setup Flow
1. You'll be redirected to the **How would you like to build your model?** screen. Choose between building a **Custom AI** from scratch or importing a pre-existing model. Select **Build** **custom AI** to continue.
![Screenshot 2026 05 28 At 1 53 15 PM]( "Screenshot 2026 05 28 At 1 53 15 PM")
![Screenshot 2026 05 28 At 1 53 15 PM]( "Screenshot 2026 05 28 At 1 53 15 PM")
2. On the next screen, provide details about how you plan to use your AI model so the platform can tailor its recommendations to your needs. Fill in the following fields:
* **Role** (dropdown) — Select your job role or title. This helps the platform understand your technical background and personalize the setup experience accordingly.
* **Company** (text field) — Enter the name of your organization or company.
* **I want an AI for...** (dropdown) — Select the primary purpose or task you want your AI model to perform (e.g., customer support, data analysis, content generation).
* **Using...** (dropdown) — Select the type of data or content your model will work with. Once all fields are completed, proceed to the next step. This information is used to guide configuration recommendations and does not limit your options later.
![Screenshot 2026 05 28 At 2 05 36 PM]( "Screenshot 2026 05 28 At 2 05 36 PM")
![Screenshot 2026 05 28 At 2 05 36 PM]( "Screenshot 2026 05 28 At 2 05 36 PM")
3. This screen is where you provide the data your AI model will learn from. You can supply your data in three ways, selectable via the tabs at the top:
* **Upload Files** (default tab) — Click **Choose a File** to browse and upload files from your device. Multiple files are supported.
* [**Integrations**](https://docs.4minds.ai/integrations) (tab) — Connect to an external data source or cloud storage integration instead of uploading files manually.
* **URL** (tab) — Provide a web URL to pull content from directly. Alternatively, at the bottom of the screen you can select **Use an existing dataset** from the dropdown if you have previously uploaded data you'd like to reuse. Click the '**Upload Files**' button to proceed.
![Screenshot 2026 05 28 At 2 19 39 PM]( "Screenshot 2026 05 28 At 2 19 39 PM")
![Screenshot 2026 05 28 At 2 19 39 PM]( "Screenshot 2026 05 28 At 2 19 39 PM")
4. Once your model is set up, you interact with it in the **Personalized Chat Screen**. Key elements include:
* **Sidebar** — Create new models, personas, or chat sessions, and access Feedback and Symi Beta.
* **Top Right dropdown** — Shows and switches your active model.
* **Chat Input** — Type your prompt and use the **Web**, **Dev**, and **Image** toggles to enhance your query. Select a persona from the **None** dropdown, or enable **Advanced Mode** for greater control.
Sample prompts to get started:
* "Explain the authentication flow for our trading platform API."
* "What endpoints are available for user account management?"
* "How should a developer handle a 401 Unauthorized error?"
![Screenshot 2026 05 28 At 2 22 46 PM]()
![Screenshot 2026 05 28 At 2 22 46 PM]()
## Next steps
Explore what else is possible with 4MINDS.
Discover the full range of 4MINDS Control Center functionality
Learn how to fine-tune models using Hugging Face datasets like FinQA
# Amazon RDS
Source: https://docs.4minds.ai/rds
Connect 4MINDS to Amazon RDS to browse databases, preview tables, and import relational data as datasets — using IAM Role Federation or Amazon Cognito for AWS auth and RDS IAM Database Authentication for SQL access.
This guide walks you through connecting 4MINDS to Amazon RDS. The connection has **two layers**:
1. **AWS-level authentication** — how 4MINDS discovers which RDS instances exist in your account. Uses **IAM Role Federation** (recommended) or **Amazon Cognito**, both shared with the rest of 4MINDS's AWS integrations.
2. **Database-level authentication** — uses **RDS IAM Database Authentication** to generate short-lived tokens from the same AWS credentials used for instance discovery. No separate database username or password is required.
This two-layer model lets 4MINDS list your RDS fleet and connect to individual databases with a single set of AWS credentials — no long-lived database passwords to manage or rotate.
For the generic IAM Role / Cognito setup (creating the OIDC provider, trust policy, Cognito user pool, etc.), see [AWS Integrations](/aws-integrations). The steps below focus on what's **specific to RDS**.
***
## Supported Database Engines
4MINDS supports database browsing on these RDS engines:
| Engine family | RDS `Engine` values |
| ------------------- | -------------------------------------------- |
| **PostgreSQL** | `postgres`, `aurora-postgresql` |
| **MySQL / MariaDB** | `mysql`, `mariadb`, `aurora-mysql`, `aurora` |
Other RDS engines (Oracle, SQL Server, etc.) can still be **listed** in the instance picker, but the database/table browser will return an "unsupported engine" error when you open them.
***
## RDS IAM Permissions Policy
Whichever AWS auth method you use (IAM Role Federation or Cognito), attach this policy to the role:
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "RDSInstanceDiscovery",
"Effect": "Allow",
"Action": [
"rds:DescribeDBInstances"
],
"Resource": "*"
},
{
"Sid": "RDSIAMDatabaseAuth",
"Effect": "Allow",
"Action": "rds-db:connect",
"Resource": "arn:aws:rds-db:*:*:dbuser:*/*"
},
{
"Sid": "IdentityVerification",
"Effect": "Allow",
"Action": "sts:GetCallerIdentity",
"Resource": "*"
}
]
}
```
| Permission | Purpose |
| ------------------------- | ----------------------------------------------------------------------------------------------------------- |
| `rds:DescribeDBInstances` | Lists RDS instances in the connected region so 4MINDS can populate the instance picker |
| `rds-db:connect` | Generates short-lived authentication tokens for direct database connections via IAM Database Authentication |
| `sts:GetCallerIdentity` | Verifies the connection during **Test Connection** |
Name the policy something memorable like `4MINDS-RDS-Access` — you'll attach it when creating the IAM role (for Role Federation) or the Cognito authenticated role.
> **Note:** The `rds-db:connect` resource ARN above uses a wildcard for all DB users and instances. To restrict access to specific instances or database users, narrow the resource ARN (e.g., `arn:aws:rds-db:us-east-1:123456789012:dbuser:db-ABCDEFG/my_iam_user`).
### Least-Privilege: Restricting to Specific Instances
If you want to expose only a subset of your RDS fleet, scope both permissions by ARN:
```json theme={null}
{
"Effect": "Allow",
"Action": "rds:DescribeDBInstances",
"Resource": [
"arn:aws:rds:us-east-1:123456789012:db:prod-analytics",
"arn:aws:rds:us-east-1:123456789012:db:prod-reporting"
]
},
{
"Effect": "Allow",
"Action": "rds-db:connect",
"Resource": [
"arn:aws:rds-db:us-east-1:123456789012:dbuser:db-ABCDEFG/fourminds_iam_user",
"arn:aws:rds-db:us-east-1:123456789012:dbuser:db-HIJKLMN/fourminds_iam_user"
]
}
```
### RDS Instance Prerequisites for IAM Database Authentication
Each RDS instance you want 4MINDS to browse must have IAM Database Authentication enabled, and a database user configured for IAM auth:
**PostgreSQL:**
```sql theme={null}
CREATE USER fourminds_iam_user WITH LOGIN;
GRANT rds_iam TO fourminds_iam_user;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO fourminds_iam_user;
```
**MySQL / MariaDB:**
```sql theme={null}
CREATE USER 'fourminds_iam_user'@'%' IDENTIFIED WITH AWSAuthenticationPlugin AS 'RDS';
GRANT SELECT ON *.* TO 'fourminds_iam_user'@'%';
```
To enable IAM Database Authentication on an existing instance, see [AWS documentation — Enabling and disabling IAM database authentication](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.Enabling.html).
***
## Connection Method 1: IAM Role Federation (Recommended)
Short-lived credentials minted per request through AWS STS — no long-lived AWS keys stored in 4MINDS.
1. Complete the one-time AWS setup in [AWS Integrations → IAM Role Federation](/aws-integrations#connection-method-1-iam-role-federation-recommended) (OIDC provider + IAM role with trust policy).
2. Attach the **RDS IAM policy** above to the role (Role Federation and Cognito can share the same role — just attach both service policies to it if you use multiple AWS integrations).
3. In 4MINDS, open **Integrations** → **Amazon RDS** → click **Configure** (or **Settings** if already connected).
4. Keep the **IAM Role** tab selected.
5. Paste your **IAM Role ARN** (e.g., `arn:aws:iam::123456789012:role/4MINDS-integration-role`).
6. Leave the **External ID** field blank — it is not supported for IAM Role Federation (see [AWS Integrations → Verify the Trust Policy](/aws-integrations#c-verify-the-trust-policy)).
7. Enter your **AWS Region** (e.g., `us-east-1`) — this must match the region of the RDS instances you want to browse.
8. Click **Test Connection** — success shows *"Connection successful! Found N instance(s)."*
9. Click **Save Credentials**.
***
## Connection Method 2: Amazon Cognito
Use this if your organization already authenticates against AWS through Cognito User Pools and Identity Pools.
1. Complete the one-time Cognito setup in [AWS Integrations → Amazon Cognito](/aws-integrations#connection-method-2-amazon-cognito) (User Pool, App Client, Identity Pool, authenticated role).
2. Attach the **RDS IAM policy** above to the Cognito **authenticated role**.
3. In 4MINDS, open **Integrations** → **Amazon RDS** → click **Configure**.
4. Switch to the **Cognito** tab.
5. Fill in **User Pool ID**, **App Client ID**, **App Client Secret** (if configured), **Identity Pool ID**, **Username**, and **Password**.
6. Enter your **AWS Region**.
7. Click **Test Connection**, then **Save Credentials**.
***
## 4MINDS Fields
| Field | Required | Notes |
| ------------------ | --------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| **AWS Region** | Yes | Region where your RDS instances are deployed. Must match the region of your Cognito resources (Cognito method) |
| **IAM Role ARN** | IAM Role method | Role with the RDS IAM policy attached |
| **External ID** | No | Not supported for IAM Role Federation — leave blank. `sts:ExternalId` conditions are not accepted by `AssumeRoleWithWebIdentity` |
| **Cognito fields** | Cognito method | See [AWS Integrations](/aws-integrations#gather-your-cognito-details) |
***
## Importing Data from RDS
Once connected, open the **Datasets** page and click **Add Source** → **Amazon RDS** (or pick RDS from the data source bar inside an existing dataset). The import wizard walks you through four steps:
### Step 1 — Pick an Instance
4MINDS calls `rds:DescribeDBInstances` in your configured region and shows every instance the role can see, with its engine, engine version, instance class, and status. Use the search box to filter by identifier or engine. Click an instance to continue.
> Instances not in the `available` state are shown with a warning color and may fail to connect in Step 2.
### Step 2 — Connect to the Database
4MINDS uses the same AWS credentials from your integration to generate a short-lived IAM authentication token via `generate_db_auth_token`. No separate database username or password is needed — the connection is established automatically using IAM Database Authentication.
* Authentication tokens are generated per-request and expire after 15 minutes.
* The IAM DB user configured on the instance (see [RDS Instance Prerequisites](#rds-instance-prerequisites-for-iam-database-authentication)) determines what permissions the connection has.
* All connections use SSL, as required by RDS IAM Database Authentication.
### Step 3 — Browse Databases
4MINDS opens a live SQL connection to the instance endpoint using the IAM-generated token and lists user databases. System databases (`rdsadmin`, `mysql`, `information_schema`, `performance_schema`, `sys`) are filtered out.
### Step 4 — Select Tables
Click a database to list its base tables. For each table the browser shows:
* Schema-qualified name (e.g., `public.customers` for Postgres, or just `orders` for MySQL)
* Estimated row count (from `pg_stat_user_tables` / `information_schema.tables`)
* Column count
* On-disk size
Check the tables you want to import and click **Add Tables**. Selected tables are staged into your dataset alongside files from any other source.
### During Import
At import time, 4MINDS runs a bulk fetch on each selected table:
* Up to **50,000 rows** per table (hard cap — larger tables are truncated and flagged).
* **Binary columns** (`BYTEA`, `BLOB`, `VARBINARY`, etc.) are dropped from the export rather than base64-encoded, to keep CSV output reasonable.
* Complex types (JSONB, arrays) are serialized with `json.dumps` into a single cell.
* Timeout is **120 seconds** per table.
Each table lands in your dataset as a UTF-8 CSV with the original column names as headers.
***
## Networking Requirements
RDS instances live inside a VPC. For 4MINDS to open a SQL connection, the instance must be reachable from the 4MINDS backend's outbound network:
* **Security group inbound rule** — allow the database port (5432 for Postgres, 3306 for MySQL/MariaDB) from the 4MINDS backend's public IP or the CIDR range of its egress.
* **Publicly accessible = Yes** is the simplest path for temporary testing; for production, prefer a VPC peering / PrivateLink setup with 4MINDS support.
* **SSL** — All connections use SSL, as required by RDS IAM Database Authentication. Ensure your RDS instance has a valid SSL certificate (AWS-managed certificates work out of the box).
If the SQL connection times out, the browser shows:
> *"Connection timed out. The backend may not be able to reach the RDS endpoint — check VPC / security group rules (inbound port open from the backend's network)."*
***
## Security Model
| Credential | Where it lives | How long |
| --------------------------------- | ------------------------------------ | -------------------------------- |
| AWS IAM Role ARN / Cognito config | 4MINDS database (encrypted metadata) | Until you disconnect |
| Temporary AWS credentials (STS) | Request memory only | \~1 hour, minted per call |
| IAM DB authentication token | Request memory only | \~15 minutes, generated per call |
No database passwords are stored or transmitted. Each SQL endpoint (`/instances/{id}/databases`, `/tables`, `/preview`) generates a short-lived IAM authentication token from the existing AWS credentials, opens a short-lived SQLAlchemy engine over SSL, runs one query, and disposes of the engine before returning.
***
## Testing Your Connection
After saving credentials, the **Test Connection** button:
1. Resolves your auth method → AWS credentials (via STS AssumeRoleWithWebIdentity for Role Federation, or Cognito GetCredentialsForIdentity for Cognito).
2. Calls `rds:DescribeDBInstances` with a 15-second timeout.
3. Returns *"Connection successful! Found N instance(s)."* on success.
Database-level IAM authentication is tested when you select an instance in the import wizard — it uses the same AWS credentials to generate an authentication token and verify SQL connectivity.
***
## Troubleshooting
| Issue | Solution |
| ----------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `AccessDenied` on Test Connection | The IAM policy isn't attached to the role, or is missing `rds:DescribeDBInstances` |
| `Invalid Access Key ID` / `Invalid Secret Access Key` | Wrong static credentials (only relevant if using legacy access-keys auth) |
| `Unrecognized client` | Wrong AWS region, or RDS isn't enabled in that region |
| `Could not resolve RDS endpoint hostname` | The instance endpoint in the instance list is wrong, or DNS can't resolve it from the backend |
| `Connection timed out` (SQL step) | Security group doesn't allow inbound from the 4MINDS backend's egress IP/CIDR |
| `Connection refused` | Port isn't open on the security group, or the instance isn't `Publicly accessible` and there's no peering |
| `Authentication failed` | IAM Database Authentication is not enabled on the instance, the IAM DB user doesn't exist, or the IAM policy is missing `rds-db:connect`. See [RDS Instance Prerequisites](#rds-instance-prerequisites-for-iam-database-authentication) |
| `SSL error connecting to RDS` | IAM DB auth requires SSL — ensure the instance has a valid certificate. AWS-managed certificates work out of the box |
| `RDS engine 'oracle-ee' is not supported` | Database browsing only works on Postgres / MySQL / MariaDB / Aurora variants. See [Supported Database Engines](#supported-database-engines) |
| Tables list is empty | The database has no base tables, or the master user lacks `SELECT` on `information_schema` / `pg_stat_user_tables` |
***
## Disconnecting
To remove the RDS integration:
1. Open **Integrations** → **Amazon RDS** → click **Settings**.
2. Click **Disconnect**.
This deactivates the stored AWS connection metadata on 4MINDS. Your AWS resources (IAM roles, OIDC providers, Cognito pools, policies, and IAM DB users) are untouched — delete them in the AWS Console if they're no longer needed.
# Amazon Redshift
Source: https://docs.4minds.ai/redshift
Connect 4MINDS to Amazon Redshift — provisioned clusters and Redshift Serverless — to browse databases, preview tables, and import data as datasets, using IAM Role Federation or Amazon Cognito for AWS auth and temporary Redshift credentials for SQL access.
This guide walks you through connecting 4MINDS to Amazon Redshift. 4MINDS supports both **provisioned clusters** and **Redshift Serverless workgroups** — they appear together in the same picker. The connection has **two layers**:
1. **AWS-level authentication** — how 4MINDS discovers which Redshift clusters and Serverless workgroups exist in your account. Uses **IAM Role Federation** (recommended) or **Amazon Cognito**, both shared with the rest of 4MINDS's AWS integrations.
2. **Database-level authentication** — uses **temporary Redshift credentials** minted from the same AWS credentials used for discovery. Provisioned clusters use `redshift:GetClusterCredentials`; Serverless workgroups use `redshift-serverless:GetCredentials`. No long-lived database password is stored.
This two-layer model lets 4MINDS list your Redshift resources and connect to individual databases with a single set of AWS credentials — no long-lived database passwords to manage or rotate.
For the generic IAM Role / Cognito setup (creating the OIDC provider, trust policy, Cognito user pool, etc.), see [AWS Integrations](/aws-integrations). The steps below focus on what's **specific to Redshift**.
***
## Provisioned vs. Serverless
| | Provisioned cluster | Redshift Serverless |
| ------------------ | ------------------------------------------------------------ | ------------------------------------------------------------------------- |
| Discovery API | `redshift:DescribeClusters` | `redshift-serverless:ListWorkgroups` |
| Credential API | `redshift:GetClusterCredentials` | `redshift-serverless:GetCredentials` |
| Database user | You choose the DB user (see [Database User](#database-user)) | Derived automatically from the IAM identity — **no DB user to configure** |
| Default port | `5439` | `5439` |
| Shown in picker as | Node type + node count | `Serverless · ` |
Both types are listed together when you browse Redshift in the import wizard. The difference is handled server-side; you don't have to pick a mode.
***
## Redshift IAM Permissions Policy
Whichever AWS auth method you use (IAM Role Federation or Cognito), attach this policy to the role. It covers **both** provisioned and Serverless. If you only use one, you can drop the statements for the other.
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "RedshiftProvisionedDiscovery",
"Effect": "Allow",
"Action": [
"redshift:DescribeClusters"
],
"Resource": "*"
},
{
"Sid": "RedshiftProvisionedCredentials",
"Effect": "Allow",
"Action": "redshift:GetClusterCredentials",
"Resource": [
"arn:aws:redshift:*:*:dbuser:*/*",
"arn:aws:redshift:*:*:dbname:*/*"
]
},
{
"Sid": "RedshiftServerlessDiscovery",
"Effect": "Allow",
"Action": [
"redshift-serverless:ListWorkgroups"
],
"Resource": "*"
},
{
"Sid": "RedshiftServerlessCredentials",
"Effect": "Allow",
"Action": "redshift-serverless:GetCredentials",
"Resource": "*"
}
]
}
```
| Permission | Purpose |
| ------------------------------------ | --------------------------------------------------------------------------------------------------------------------- |
| `redshift:DescribeClusters` | Lists provisioned clusters in the connected region so 4MINDS can populate the picker (also backs **Test Connection**) |
| `redshift:GetClusterCredentials` | Mints short-lived DB credentials for a provisioned cluster + DB user |
| `redshift-serverless:ListWorkgroups` | Lists Serverless workgroups in the connected region |
| `redshift-serverless:GetCredentials` | Mints short-lived DB credentials for a Serverless workgroup |
Name the policy something memorable like `4MINDS-Redshift-Access` — you'll attach it when creating the IAM role (for Role Federation) or the Cognito authenticated role.
> **Note:** `GetClusterCredentials` is authorized against both the `dbuser:` and `dbname:` resource ARNs. The wildcards above allow all clusters, DB users, and databases. To restrict, narrow the ARNs (e.g., `arn:aws:redshift:us-east-1:123456789012:dbuser:my-cluster/fourminds_iam_user`).
### Least-Privilege: Restricting to Specific Resources
To expose only a subset of your Redshift resources, scope the discovery and credential ARNs:
```json theme={null}
{
"Effect": "Allow",
"Action": "redshift:GetClusterCredentials",
"Resource": [
"arn:aws:redshift:us-east-1:123456789012:dbuser:prod-analytics/fourminds_iam_user",
"arn:aws:redshift:us-east-1:123456789012:dbname:prod-analytics/dev"
]
},
{
"Effect": "Allow",
"Action": "redshift-serverless:GetCredentials",
"Resource": "arn:aws:redshift-serverless:us-east-1:123456789012:workgroup/*"
}
```
***
## Database Access & Grants
The IAM identity 4MINDS assumes must map to a database user that has read access to the tables you want to import.
> **How 4MINDS decides what you can browse:** databases, schemas, and tables are listed through Redshift's grant-aware system views (`SVV_ALL_SCHEMAS`, `SVV_ALL_TABLES`, `SVV_ALL_COLUMNS`). A schema or table appears **only if the resolved DB user has been granted access to it** — the user does **not** need to *own* it. Grant `USAGE` on each schema **and** `SELECT` on its tables (below) for **every** schema you want to browse, not just `public`; a schema the user has no grant on is silently omitted from the picker, and a table without a `SELECT` grant won't appear even if its schema does. The same grants make external (Redshift Spectrum) and datashare tables browsable once they're accessible to the user.
### Provisioned clusters — the DB user
For a provisioned cluster, 4MINDS calls `GetClusterCredentials` with a **DB username** you configure (see [Database User](#database-user)). That user must already exist in the cluster and have `SELECT` on the target tables. `AutoCreate` is **not** used — 4MINDS never creates users in your cluster.
```sql theme={null}
-- run in the target database on your Redshift cluster
CREATE USER fourminds_iam_user PASSWORD DISABLE;
GRANT USAGE ON SCHEMA public TO fourminds_iam_user;
-- standing rule: covers existing AND future tables/views in the schema,
-- regardless of which role creates them
GRANT SELECT FOR TABLES IN SCHEMA public TO fourminds_iam_user;
```
> **Older Redshift versions:** if `GRANT ... FOR TABLES IN SCHEMA` (scoped privileges) isn't
> available on your cluster, use the equivalent snapshot pattern instead — but remember it
> only covers objects that exist now, so re-run it (or set default privileges per creating
> role) as pipelines add tables:
>
> ```sql theme={null}
> GRANT SELECT ON ALL TABLES IN SCHEMA public TO fourminds_iam_user;
> ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO fourminds_iam_user;
> ```
The IAM policy's `dbuser:` ARN must match this username (e.g., `.../my-cluster/fourminds_iam_user`).
> **Important — why newly-created tables can go missing (most common cause of "most of my tables don't show up"):**
>
> * **`GRANT SELECT ON ALL TABLES IN SCHEMA` is a one-time snapshot.** It grants access only to the objects that exist **at the moment you run it**. Any table, view, or materialized view created *afterward* is **not** covered and will silently be omitted from the browser until you re-grant. If a pipeline (e.g. dbt, an ETL job, or a nightly build) keeps adding objects to a schema, you'll see the older objects but not the newer ones.
> * **`ALTER DEFAULT PRIVILEGES` only applies to objects created by the role that runs it, going forward.** Default privileges are scoped to the *creating* role. If your tables are created by a **different role** than the one that ran the `ALTER DEFAULT PRIVILEGES` statement (very common — e.g. tables created by a `dbt_service` role while you granted as an admin), the default grant does **not** apply and new objects stay invisible.
>
> **Fix:** run `ALTER DEFAULT PRIVILEGES` **as each role that creates objects** in the schema. For example, if dbt writes as role `dbt_service`:
>
> ```sql theme={null}
> -- run while connected as (or SET SESSION AUTHORIZATION to) the creating role
> ALTER DEFAULT PRIVILEGES FOR ROLE dbt_service IN SCHEMA dw_mdl
> GRANT SELECT ON TABLES TO fourminds_iam_user;
> ```
>
> To immediately catch up existing objects the read user is still missing, re-run
> `GRANT SELECT ON ALL TABLES IN SCHEMA TO fourminds_iam_user;` once now.
> **Simpler fix — grant a standing rule for the whole schema.** Redshift's scoped-privileges
> syntax grants `SELECT` on **existing *and* future** tables and views in a schema in a single
> statement, regardless of which role creates them — so it avoids both the `ALL TABLES` snapshot
> gap and the creator-scoped limitation of `ALTER DEFAULT PRIVILEGES`:
>
> ```sql theme={null}
> GRANT SELECT FOR TABLES IN SCHEMA dw_mdl TO fourminds_iam_user;
> ```
>
> This is the recommended one-shot fix when a customer reports that only *some* tables in a
> schema show up: it backfills the objects the read user is missing today and keeps new
> objects visible as pipelines add them, without having to run `ALTER DEFAULT PRIVILEGES` per
> creating role.
### Serverless workgroups — IAM-derived user
Redshift Serverless `GetCredentials` does **not** accept a DB username — it derives the database user from the IAM identity making the call (returned as `IAMR:` or `IAM:`). To grant it access, create/grant that exact user in the database:
```sql theme={null}
-- is the IAM role 4MINDS assumes (Role Federation) or the
-- Cognito authenticated role. Redshift prefixes assumed roles with "IAMR:".
CREATE USER "IAMR:4MINDS-integration-role" PASSWORD DISABLE;
GRANT USAGE ON SCHEMA public TO "IAMR:4MINDS-integration-role";
-- standing rule: covers existing AND future tables/views (see provisioned note above
-- for the ON ALL TABLES + ALTER DEFAULT PRIVILEGES fallback on older Redshift versions)
GRANT SELECT FOR TABLES IN SCHEMA public TO "IAMR:4MINDS-integration-role";
```
> **Tip:** If you're unsure of the exact derived username, run a browse in 4MINDS and check the error message — a `permission denied` failure will name the DB user Redshift resolved, which you can then `GRANT` to.
> **Note:** For Serverless, `GetCredentials` **auto-creates** the `IAMR:` database user on first use, so the explicit `CREATE USER` above is optional — granting `SELECT` to `PUBLIC` (or to the resolved user after a first browse) is sufficient. Pre-creating the user is only needed if you want to grant privileges before the first connection.
### Views (regular, late-binding, materialized)
4MINDS lists and imports **views** alongside tables — regular views, late-binding views (`WITH NO SCHEMA BINDING`), and materialized views all appear in the browser and import through the same reflect-and-`SELECT` path a table uses.
Grant behavior (verified on Redshift):
* `GRANT SELECT FOR TABLES IN SCHEMA ` (and the older `GRANT SELECT ON ALL TABLES IN SCHEMA `) **cover all three view types** as well as tables — no separate grant is required for views or materialized views.
* The standing-rule `FOR TABLES IN SCHEMA` grant covers **future** views and materialized views automatically. If you use the snapshot `ON ALL TABLES` form instead, pair it with `ALTER DEFAULT PRIVILEGES IN SCHEMA GRANT SELECT ON TABLES` (per creating role) so view-backed datasets stay browsable as they're added.
* Reading a view needs `SELECT` on the **view itself only** — the DB user does **not** need `SELECT` on the view's underlying base tables. This holds for regular, late-binding, and materialized views.
* Browsing is consistent with preview: if a view appears in the picker, the resolved user can preview and import it. There's no "shows in the list but fails to import" gap from grants.
If you grant **table-by-table** instead of using `ALL TABLES`, remember to include the specific views and materialized views you want browsable — otherwise they're silently omitted just like an ungranted table:
```sql theme={null}
GRANT SELECT ON . TO "";
GRANT SELECT ON . TO "";
```
***
## Connection Method 1: IAM Role Federation (Recommended)
Short-lived credentials minted per request through AWS STS — no long-lived AWS keys stored in 4MINDS.
1. Complete the one-time AWS setup in [AWS Integrations → IAM Role Federation](/aws-integrations#connection-method-1-iam-role-federation-recommended) (OIDC provider + IAM role with trust policy).
2. Attach the **Redshift IAM policy** above to the role (Role Federation and Cognito can share the same role — just attach both service policies to it if you use multiple AWS integrations).
3. In 4MINDS, open **Integrations** → **Amazon Redshift** → click **Configure** (or **Settings** if already connected).
4. Keep the **IAM Role** tab selected.
5. Paste your **IAM Role ARN** (e.g., `arn:aws:iam::123456789012:role/4MINDS-integration-role`).
6. Leave the **External ID** field blank — it is not supported for IAM Role Federation (see [AWS Integrations → Verify the Trust Policy](/aws-integrations#c-verify-the-trust-policy)).
7. Enter your **AWS Region** (e.g., `us-east-1`) — this must match the region of the Redshift resources you want to browse.
8. *(Provisioned only)* Enter a **Database user** — see [Database User](#database-user). Leave blank to use the default `fourminds_iam_user`.
9. Click **Test Connection** — success shows *"Connection successful. Found N Redshift cluster(s)."*
10. Click **Save Credentials**.
***
## Connection Method 2: Amazon Cognito
Use this if your organization already authenticates against AWS through Cognito User Pools and Identity Pools.
1. Complete the one-time Cognito setup in [AWS Integrations → Amazon Cognito](/aws-integrations#connection-method-2-amazon-cognito) (User Pool, App Client, Identity Pool, authenticated role).
2. Attach the **Redshift IAM policy** above to the Cognito **authenticated role**.
3. In 4MINDS, open **Integrations** → **Amazon Redshift** → click **Configure**.
4. Switch to the **Cognito** tab.
5. Fill in **User Pool ID**, **App Client ID**, **App Client Secret** (if configured), **Identity Pool ID**, **Username**, and **Password**.
6. Enter your **AWS Region**.
7. *(Provisioned only)* Enter a **Database user** if you're not using the default.
8. Click **Test Connection**, then **Save Credentials**.
***
## Database User
`GetClusterCredentials` (provisioned clusters only) authenticates as a specific database user. In the settings modal, the **Database user** field controls which user 4MINDS requests credentials for:
* Defaults to `fourminds_iam_user` if left blank.
* The user **must already exist** in the cluster with `SELECT` on the tables you want to import (see [Database Access & Grants](#database-access--grants)).
* The value is stored once with your connection metadata and reused for every browse and import — you don't re-enter it per request.
* **Ignored for Serverless** — Serverless derives the user from the IAM identity, so this field has no effect on workgroups.
***
## 4MINDS Fields
| Field | Required | Notes |
| ------------------ | --------------- | ------------------------------------------------------------------------------------------------------------------- |
| **AWS Region** | Yes | Region where your Redshift resources are deployed. Must match the region of your Cognito resources (Cognito method) |
| **IAM Role ARN** | IAM Role method | Role with the Redshift IAM policy attached |
| **External ID** | No | Not supported for IAM Role Federation — leave blank |
| **Database user** | No | Provisioned clusters only. Defaults to `fourminds_iam_user`. Ignored for Serverless |
| **Cognito fields** | Cognito method | See [AWS Integrations](/aws-integrations#gather-your-cognito-details) |
***
## Importing Data from Redshift
Once connected, open the **Datasets** page and click **Add Source** → **Amazon Redshift** (or pick Redshift from the data source bar inside an existing dataset). The import wizard walks you through the hierarchy: **clusters/workgroups → databases → tables**.
### Step 1 — Pick a Cluster or Workgroup
4MINDS calls `redshift:DescribeClusters` **and** `redshift-serverless:ListWorkgroups` in your configured region and shows everything the role can see. Provisioned clusters show their node type and node count; Serverless workgroups are labeled `Serverless` with their namespace. Click one to continue.
> If the Serverless permissions are missing, workgroups are simply omitted — provisioned clusters still list normally.
### Step 2 — Browse Databases
4MINDS mints a short-lived credential (via `GetClusterCredentials` for a cluster, or `GetCredentials` for a workgroup), opens a live SQL connection to the endpoint over SSL, and lists databases through the grant-aware `SVV_REDSHIFT_DATABASES` catalog. This surfaces every database the resolved DB user can access — including **datashare (shared) databases provided by other clusters or accounts** — not just local databases. System databases (`template0`, `template1`, `padb_harvest`, `rdsadmin`, and the non-connectable `sys:internal`) are filtered out.
### Step 3 — Select Tables
Click a database to list its tables **and views** — local, external/Spectrum, and datashare relations the DB user can access, including **regular, late-binding, and materialized views**. For each relation the browser shows:
* Schema-qualified name (e.g., `public.customers`)
* Column count
* On-disk size (when available)
Check the tables you want to import and click **Add Tables**. Selected tables are staged into your dataset alongside files from any other source.
> **Note:** Listing of **views** (regular, late-binding, and materialized) alongside tables requires the current 4MINDS release. Earlier releases listed base tables only; if views are missing from the browser and you've confirmed the grants above, verify you're on the latest release.
### During Import
At import time, 4MINDS runs a bulk fetch on each selected table:
* Up to **50,000 rows** per table (hard cap — larger tables are truncated and flagged).
* **Binary columns** (`BYTEA`, `VARBINARY`, `BINARY`) are dropped from the export rather than base64-encoded.
* Complex types (arrays, structured values) are serialized with `json.dumps` into a single cell.
* Timeout is **120 seconds** per table.
Each table lands in your dataset as a UTF-8 CSV with the original column names as headers.
***
## Dataset Sync
### Overview
Beyond one-time imports, 4MINDS can keep a Redshift-backed dataset up to date automatically. When you enable **Dataset Sync** on a dataset built from Redshift tables, 4MINDS re-exports the configured tables on a schedule — no manual re-imports.
Unlike file-based sources (S3, Dropbox, Databricks volumes), Redshift is a **table warehouse** with no cheap per-table "last modified" signal. So Redshift sync follows the same **full-snapshot** model as BigQuery rather than incremental file diffing: **each sync cycle re-exports the whole table** (up to the 50,000-row cap) and replaces the previous snapshot. There is no row-level change detection — every run is a fresh point-in-time copy.
### How to Set Up Sync
1. Import one or more Redshift tables into a 4MINDS dataset (using the import wizard above).
2. On the dataset, toggle **Dataset Sync** on.
3. Select a sync frequency (see table below).
4. From that point on, 4MINDS re-exports each configured table at the chosen interval.
A single sync configuration can span **multiple clusters, workgroups, and databases** — each table carries its own connection target (endpoint, port, serverless flag, schema), so tables from different Redshift sources can live in the same synced dataset.
### Sync Frequencies
| Frequency | Interval | Best For |
| ---------------- | -------- | -------------------------------------------------------- |
| **Every minute** | 1 minute | Rapidly changing tables (Enterprise tier) |
| **Hourly** | 1 hour | Frequently updated warehouse tables (Teams & Enterprise) |
| **Daily** | 24 hours | Standard business reporting (All paid tiers) |
| **Weekly** | 7 days | Slowly changing reference data |
| **Monthly** | 30 days | Compliance snapshots, archival data |
> Because every Redshift sync is a full-table re-export, prefer **less frequent** intervals for large tables to control export cost and RPU/credit usage. Daily or weekly is a good default for warehouse tables.
### How It Works Internally
**Full snapshot, every cycle:** Because Redshift exposes no per-table modification timestamp, the sync manifest records no `modified_at` for Redshift entries. As a result, the writer treats every configured table as changed on every run and re-downloads a fresh snapshot — the same semantic BigQuery uses.
**Stable table identity:** Each table is tracked by a stable identifier of the form `rs----` (schema defaults to `public`). The manifest de-duplicates on this id, so re-running a sync replaces the prior snapshot of a table rather than accumulating duplicates.
**Export pipeline:** On each cycle, for every configured table 4MINDS mints a short-lived Redshift credential (via `GetClusterCredentials` for a cluster, or `GetCredentials` for a workgroup), opens an SSL SQL connection, and exports the table to CSV bytes — up to **50,000 rows** (the same cap as one-shot import; larger tables are truncated and flagged). Binary columns are dropped and empty tables are skipped. The CSV is uploaded to the user's dataset storage and processed through the 4MINDS ETL pipeline.
**Concurrent processing:** As with other sources, multiple dataset syncs run in parallel, each with its own isolated database session.
### Authentication for Automated Sync
Dataset sync runs in the background with no user present, so 4MINDS replays the **stored AWS connection** to authenticate each cycle — exactly the same two-layer model used for interactive browsing:
1. The stored auth method (IAM Role Federation or Cognito) is resolved into **fresh temporary AWS credentials** for the run (STS `AssumeRoleWithWebIdentity`, or Cognito `GetCredentialsForIdentity`).
2. Those credentials mint a **fresh temporary Redshift credential** per table (`GetClusterCredentials` / `GetCredentials`).
No long-lived database password is ever stored. The IAM role (or Cognito authenticated role) must retain the [Redshift IAM policy](#redshift-iam-permissions-policy), and the resolved DB user must keep `SELECT` on the synced tables (see [Database Access & Grants](#database-access--grants)) — a revoked grant surfaces as a skipped table in the sync logs.
***
## Networking Requirements
Redshift clusters and Serverless workgroups live inside a VPC. Only the **4MINDS backend** opens SQL connections to Redshift — the browser app (`app.4minds.ai`) never connects to your database directly, so you only need to allow the **backend's egress IP**. There is no frontend IP to allowlist.
For 4MINDS to open a SQL connection, the endpoint must be reachable from the backend's outbound network:
* **Allow the 4MINDS backend egress IP** on the Redshift port (default `5439`):
| Environment | Backend hostname | Egress IP to allowlist |
| ----------- | ---------------- | ---------------------- |
| Production | `api.4minds.ai` | `20.7.240.218/32` |
Add it as an inbound rule on the security group attached to your cluster or Serverless workgroup. AWS CLI:
```bash theme={null}
aws ec2 authorize-security-group-ingress \
--group-id \
--protocol tcp --port 5439 \
--cidr 20.7.240.218/32
```
Or in the console: **EC2 → Security Groups →** your Redshift SG **→ Inbound rules → Edit inbound rules → Add rule**, then set **Type** = `Custom TCP`, **Port range** = `5439`, **Source** = `20.7.240.218/32`.
* **Publicly accessible = Yes** is the simplest path for temporary testing; for production, prefer a VPC peering / PrivateLink setup with 4MINDS support.
* **SSL** — All connections use SSL (`sslmode=require`), as required for temporary-credential auth.
### Private cluster behind a load balancer (Connection host override)
Endpoint discovery (`redshift:DescribeClusters` for a provisioned cluster, `redshift-serverless:GetWorkgroup` for a Serverless workgroup) returns the source's **internal** endpoint. If your cluster or workgroup can't be made publicly accessible (for example, its VPC has no internet gateway) but you expose a reachable path in front of it — such as a Network Load Balancer (NLB) — that internal endpoint won't resolve or connect from the 4MINDS backend, and Test/browse will time out.
For this case, set the optional **Connection host** field in the Redshift connection settings to your reachable hostname (e.g. `my-cluster-nlb.elb.us-west-2.amazonaws.com`). 4MINDS then opens the SQL connection to that host instead of the discovered endpoint, while still minting temporary IAM credentials against your **Cluster identifier** (provisioned) or **Workgroup** (Serverless) — so IAM auth is unchanged. This works the same way for provisioned clusters and Serverless workgroups.
Requirements when using the override:
* The host must forward TCP `5439` to your cluster or workgroup and present its certificate (SSL is still required). A Network Load Balancer with a TCP listener passes the TLS connection straight through, so the source's own certificate reaches the backend.
* The **Cluster identifier** / **Workgroup**, **Region**, and **Database user** must still match the real source so credential minting (`GetClusterCredentials` for a cluster, `GetCredentials` for a workgroup) succeeds.
* The 4MINDS backend egress IP (`20.7.240.218/32`) must be allowed to reach the override host on port `5439`.
* One override applies per connection; it's intended for a single private cluster or workgroup reached through a fixed hostname.
> **Note:** If a connection still times out after allowlisting the IP above, the backend egress IP may have changed — contact 4MINDS support to confirm the current address before widening your rules.
If the SQL connection times out, the browser shows:
> *"Connection timed out. The backend may not be able to reach the Redshift endpoint — check VPC / security group rules (inbound port open from the backend's network)."*
***
## Security Model
| Credential | Where it lives | How long |
| --------------------------------------------------- | ------------------------------------ | ---------------------------- |
| AWS IAM Role ARN / Cognito config | 4MINDS database (encrypted metadata) | Until you disconnect |
| Temporary AWS credentials (STS) | Request memory only | \~1 hour, minted per call |
| Redshift DB credentials (`Get(Cluster)Credentials`) | Request memory only | \~1 hour, generated per call |
No database passwords are stored or transmitted. Each SQL operation mints a short-lived credential from the existing AWS credentials, opens a short-lived SQLAlchemy engine over SSL, runs one query, and disposes of the engine before returning.
***
## Testing Your Connection
After saving credentials, the **Test Connection** button:
1. Resolves your auth method → AWS credentials (via STS AssumeRoleWithWebIdentity for Role Federation, or Cognito GetCredentialsForIdentity for Cognito).
2. Calls `redshift:DescribeClusters` with a 15-second timeout.
3. Returns *"Connection successful. Found N Redshift cluster(s)."* on success.
Database-level authentication is tested when you browse a cluster or workgroup in the import wizard — it uses the same AWS credentials to mint a temporary credential and verify SQL connectivity.
***
## Troubleshooting
| Issue | Solution |
| ------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `AccessDenied` on Test Connection | The IAM policy isn't attached to the role, or is missing `redshift:DescribeClusters` |
| Serverless workgroups don't appear | The role is missing `redshift-serverless:ListWorkgroups`, or there are no workgroups in that region |
| `Invalid Access Key ID` / `Invalid Secret Access Key` | Wrong static credentials (only relevant if using legacy access-keys auth) |
| `Unrecognized client` | Wrong AWS region, or Redshift isn't enabled in that region |
| `Could not resolve Redshift endpoint hostname` | The endpoint in the list is wrong, or DNS can't resolve it from the backend |
| `Connection timed out` (SQL step) | Security group doesn't allow inbound port `5439` from the 4MINDS backend egress IP (`20.7.240.218/32`) — see [Networking Requirements](#networking-requirements) |
| `Connection refused` | Port `5439` isn't open on the security group, or the endpoint isn't reachable and there's no peering |
| `Connection timed out` on a **private** cluster or workgroup fronted by an NLB | The discovered endpoint is internal-only. Set the **Connection host** override to your reachable hostname — see [Private cluster behind a load balancer](#private-cluster-behind-a-load-balancer-connection-host-override) |
| `Authentication failed` | The IAM policy is missing `redshift:GetClusterCredentials` / `redshift-serverless:GetCredentials`, or the resolved DB user lacks a login/grant. See [Database Access & Grants](#database-access--grants) |
| `permission denied for relation ...` | The DB user exists but lacks `SELECT` on the table — run the `GRANT` statements in [Database Access & Grants](#database-access--grants) |
| `SSL error connecting to Redshift` | Temporary-credential auth requires SSL — AWS-managed certificates work out of the box |
| Tables, views, or schemas are missing from the browser | 4MINDS lists only what the resolved DB user can see via Redshift's grant-aware `SVV_ALL_*` catalog views. Grant `USAGE` on the schema **and** `SELECT` on its tables/views — for **every** schema you want to browse, not just `public`. `SELECT ON ALL TABLES IN SCHEMA` also covers views and materialized views; if you granted table-by-table, add the specific views too. See [Database Access & Grants](#database-access--grants) |
### Debugging missing tables or schemas
4MINDS populates the picker from Redshift's **grant-aware** `SVV_ALL_*` catalog views, so an object appears **only if the DB user 4MINDS connects as has been granted access to it**. When a customer connects successfully but sees fewer schemas or tables than expected, work through these checks **as the same DB user 4MINDS connects with** (`fourminds_iam_user` unless configured otherwise), in Redshift Query Editor v2, DBeaver, or `psql`.
**Step 1 — Confirm the user and database.**
```sql theme={null}
SELECT current_database(), current_user;
```
`current_user` should be the configured DB user, and `current_database()` the database the connection points at. A wrong user or database alone explains missing objects.
**Step 2 — See what this user can see in the schema, by object type.** This is what the picker shows:
```sql theme={null}
SELECT schema_name, table_type, count(*) AS object_count
FROM svv_all_tables
WHERE database_name = current_database()
AND schema_name = 'dw_mdl'
GROUP BY schema_name, table_type
ORDER BY table_type;
```
**Step 3 — Compare against the true object count (run as a superuser or the schema owner).** If this privileged total is much larger than Step 2, the gap is **grants**:
```sql theme={null}
SELECT c.relkind, -- r = table, v = view, m = materialized view, f = external
count(*) AS object_count
FROM pg_class c
JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE n.nspname = 'dw_mdl'
AND c.relkind IN ('r','v','m','f')
GROUP BY c.relkind
ORDER BY c.relkind;
```
Fix a grants gap with the standing-rule grant (covers existing and future tables and views):
```sql theme={null}
GRANT USAGE ON SCHEMA dw_mdl TO fourminds_iam_user;
GRANT SELECT FOR TABLES IN SCHEMA dw_mdl TO fourminds_iam_user;
```
**Step 4 — Rule out cross-database scoping.** Our table query is scoped to `current_database()`, so objects that resolve under a *different* database name (cross-database references, datashare-provided databases, or an external Glue/`awsdatacatalog` catalog) won't appear:
```sql theme={null}
-- what the picker sees:
SELECT count(*) FROM svv_all_tables
WHERE database_name = current_database() AND schema_name = 'dw_mdl';
-- same schema across ANY database_name:
SELECT count(*) FROM svv_all_tables
WHERE schema_name = 'dw_mdl';
```
If the second count is larger, some objects live under a different database name — point the connection at that database, or contact 4MINDS support with the datashare / external-schema details.
> **Whole schema missing vs. some tables missing:** a schema that's *entirely* absent from the picker means the user lacks `USAGE` on it (`svv_all_schemas` only returns schemas the user can enter). A schema that shows but is missing *some* tables is a `SELECT` gap — most often the `ALL TABLES` snapshot going stale as pipelines add objects. See [Database Access & Grants](#database-access--grants).
***
## Disconnecting
To remove the Redshift integration:
1. Open **Integrations** → **Amazon Redshift** → click **Settings**.
2. Click **Disconnect**.
This deactivates the stored AWS connection metadata on 4MINDS. Your AWS resources (IAM roles, OIDC providers, Cognito pools, policies, and Redshift DB users) are untouched — delete them in the AWS Console if they're no longer needed.
# Security and Compliance
Source: https://docs.4minds.ai/security
Enterprise-grade security with JWT authentication, OAuth support, encrypted data transmission, and compliance with ISO 27001, SOC-2, GDPR, and CCPA standards.
## Overview
The 4MINDS platform is built with security and privacy as core principles. Your data remains private, your models are deployed in single-tenant environments, and all transmissions are encrypted. We maintain compliance with major industry standards to meet enterprise requirements.
## Authentication
### JWT-based authentication
All API requests and platform access use JSON Web Tokens (JWT) for secure, stateless authentication. Tokens expire automatically and can be revoked instantly if needed.
### OAuth support
Sign in seamlessly with your existing accounts:
* **Google** - Use your Google Workspace or personal Google account
* **GitHub** - Authenticate with your GitHub credentials
OAuth integration eliminates the need to manage separate passwords while maintaining security through trusted identity providers.
### Enterprise Single Sign-On (SSO)
For enterprise customers, 4MINDS supports additional identity providers:
* **AWS** - Authenticate using your AWS IAM Identity Center credentials
* **Ping Identity** - Connect through your organization's Ping Identity platform
* **Okta** - Use your company's Okta identity management system
* **LDAP** - Integrate with your organization's LDAP directory service
Enterprise SSO enables centralized access control, simplified user provisioning, and compliance with your organization's security policies.
### API key generation
Generate API keys for programmatic access to your models and data. You can:
* Create multiple keys for different applications or environments
* Set expiration dates for temporary access
* Revoke keys immediately if they're compromised
* Monitor API key usage and activity
## Data privacy
### Single-tenant model deployment
Your models run in isolated, single-tenant environments. No shared infrastructure means your model weights, training data, and inference results remain completely private.
### Private data storage
All datasets, models, and generated content are stored in dedicated storage that only you can access. Data is never shared across accounts or used to train other models.
### Encrypted transmission
All data in transit uses TLS 1.3 encryption. Whether you're uploading datasets, querying models, or receiving notifications, your data is protected from interception.
## Compliance
The 4MINDS platform maintains compliance with major security and privacy regulations:
### ISO 27001
Our information security management system follows ISO 27001 standards for protecting sensitive data and managing security risks.
### SOC-2 Type II
We undergo regular SOC-2 audits to verify our security controls, availability, and confidentiality practices meet industry standards.
### GDPR
For users in the European Union, we comply with GDPR requirements for data protection, user rights, and privacy by design.
### CCPA
California users benefit from CCPA protections, including the right to know what data we collect and the right to deletion.
## MCP Security Architecture
The Model Context Protocol (MCP) enables AI systems to connect with external tools and services - but introduces significant security risks when naively integrated. Most implementations collapse decision-making, execution, and trust into the language model itself, creating opaque behavior and limited auditability.
4MINDS treats MCP as a governed system call layer, not a tool interface. Security is achieved through three independent architectural boundaries:
### Reasoning Boundary: Constellation
Constellation determines what should happen but cannot make anything happen. It doesn't open network connections, call MCP servers, hold credentials, or mutate system state. Its outputs are declarative artifacts describing intent and confidence.
This means prompt injection or model jailbreaks cannot directly trigger execution - decisions are observable and auditable before any action occurs.
### Execution Boundary: MCP Runtime
The MCP Runtime receives declarative intent and performs controlled capability invocation. It validates schemas, enforces timeouts, isolates transports, normalizes responses, and records audit logs - but never infers missing inputs or decides whether actions should occur.
This mirrors how secure operating systems handle system calls: the caller declares intent, the kernel validates and executes, and policy is enforced elsewhere.
### Governance Boundary: BAAE
All state-changing operations pass through the Bounded Autonomous Assistance Engine (BAAE), which enforces approval logic, scope limits, rate limits, kill switches, and immutable audit trails - independent of the language model.
Even a compromised MCP server cannot cause damage without crossing an explicit governance decision point.
### Transport Security
| Transport | Security Controls |
| ------------- | ------------------------------------------------------------------------------------------------------------------------- |
| **stdio** | Explicit server registration, controlled environment variables, runtime-owned process lifecycles, per-invocation auditing |
| **HTTP** | Explicit base URLs, centralized authentication, TLS requirements, timeout policies, capability whitelisting |
| **WebSocket** | Explicit connection lifecycles, message correlation, connection pool limits |
### Open Source MCP Servers
Open source MCP servers are treated as untrusted by default. Server output is normalized by the runtime, evaluated through governance, and only then reflected in user-visible outcomes. This layered trust boundary ensures unexpected server behavior cannot escalate privileges or evade audit.
## 4MINDS RBAC Capabilities Overview
### Role Hierarchy
4MINDS implements a **multi-layered** role system operating at three levels:
#### Organization Roles
| Role | Description |
| ------------------- | ---------------------------------------------------------------------------------------------------------- |
| **Owner** | Organization creator. Full permissions including delete org, manage billing, manage all teams and members. |
| **Admin** | Administrative privileges. Can manage teams, members, and settings within the organization. |
| **Billing Manager** | Handles billing operations for the organization. |
| **Member** | Standard access within the organization. |
| **Guest** | Limited, restricted access. |
#### Team Roles
| Role | Description |
| ---------- | ---------------------------------------------- |
| **Owner** | Team creator. All permissions within the team. |
| **Admin** | Administrative privileges within the team. |
| **Member** | Standard team access. |
| **Viewer** | Read-only access. |
| **Guest** | Limited guest access. |
#### System Admin Roles (Internal)
| Role | Description |
| -------------- | --------------------------------------------------------------------------------------- |
| **Superadmin** | Full platform-wide access. Bypasses all RBAC checks. Restricted to `@4minds.ai` emails. |
| **Moderator** | Support/moderation role. |
| **Support** | Customer support role. |
### Access Levels (Permission Tiers)
Users are assigned one of three **access levels** that control feature availability:
| Access Level | Capabilities |
| ------------ | ------------------------------------------------------------------------------------------------------------------------------- |
| **Basic** | Read and share datasets/models. View logs. |
| **Plus** | Everything in Basic, plus create, write, and update datasets/models. Train models. |
| **Admin** | Full access: all CRUD operations on datasets, models, teams, conversations, API keys, evaluations, inference, and integrations. |
Access levels map to subscription tiers:
* **Pro** / **Teams** / **Teams Plus** → Plus access
* **Enterprise** → Admin access
### Custom Roles and Permissions
#### Granular Permission Model
Permissions follow a `{resource}:{action}` naming convention:
**Resources:** datasets, models, teams, organizations, conversations, evaluations, inference, apikeys, logs, integrations
**Actions:** create, read, write, update, delete, train, execute, share, admin
**Permission Categories:**
* Data Management
* Model Training
* Evaluation
* Inference
* Integrations
* Organization Management
* Team Management
* Security
* Billing
#### Role-to-Permission Mapping
Permissions are mapped to roles via `RolePermission` records scoped at the organization, team, or resource level. Admins can:
* View all available permissions
* Grant or revoke individual permissions for any role
* Assign bulk access levels (basic/plus/admin) to users
#### Implicit Permission Rules
* **Delete implies all**: Having `X:delete` automatically grants `X:read`, `X:create`, `X:update`, `X:share`, `X:train`, `X:execute`
* **Owner bypass**: Organization and team owners automatically have all permissions in their scope
* **Superadmin bypass**: Superadmins bypass all permission checks platform-wide
### Resource-Level Access Control
#### Per-Resource Permissions
Beyond role-based access, 4MINDS supports **fine-grained resource-level permissions** via `UserResourcePermission` records. These can be granted to:
* **Individual users** (user\_id)
* **Entire teams** (team\_id)
* **Entire organizations** (org\_id)
Each grant includes:
* Specific resource type and ID
* Who granted it and why
* Optional expiration date
* Soft-delete via revocation timestamp
#### Model Access Control
Per-model access supports two levels:
* **Full Access** — Complete workspace, training, and inference access
* **Inference Only** — Chat/inference access only (no training, no workspace)
Per-model grants can include **restrictions**:
| Restriction Type | Details |
| ------------------- | ------------------------------------------------------------- |
| **Time Windows** | Daily start/end time, allowed weekdays, custom date schedules |
| **Date Ranges** | Valid from/until dates |
| **Token Limits** | Max tokens per session, per day |
| **Rate Limits** | Max requests per minute, hour, or day |
| **Duration Limits** | Total usage time with daily/weekly/monthly reset |
#### Dataset Access Control
Binary access model — users/teams either have access to a specific dataset or they don't. Grants include the dataset ID, grantee (user or team), and reason.
#### Resource Sharing
Datasets and models can be **shared with teams**, enabling collaborative access across the organization.
### Permission Resolution Order
When checking if a user can perform an action, 4MINDS evaluates in this order (first match wins):
1. **Superadmin** → Access granted to everything
2. **Organization Owner** → All permissions within their org
3. **Team Owner** → All permissions within their team
4. **Delete-implies-others rule** applied
5. **Resource-level permission** (highest specificity)
6. **Team-level role permission**
7. **Organization-level role permission** (lowest specificity)
### Enforcement & Frontend Integration
#### Backend Enforcement
* FastAPI dependency-injection middleware (`require_permission`, `require_organization_admin`, `require_team_member`, etc.)
* Every protected endpoint declares its required permission
* RBAC enforcement can be toggled via `RBAC_ENFORCEMENT_ENABLED` setting
#### Frontend Enforcement
* **JWT-embedded permissions** — Model access, access levels, and org privilege flags are encoded in the JWT token
* **Route guarding** — App.tsx routes users to different workspace views based on access level
* **Conditional rendering** — UI components use hooks (`useWorkspaceOnlyMode`, `useInferenceOnlyMode`, `useModelAccess`, `useFeatureGate`) to show/hide features
* **Admin portal** — Separate login and context (`AdminContext`) for internal admins
#### Audit Trail
All RBAC actions (grants, revocations, team creation, member changes) are logged in `RBACActionLog` with actor, target, resource, IP address, user agent, and timestamp.
### SSO Integration
4MINDS supports automatic role provisioning via SSO:
| Provider | Supported |
| ------------------- | --------- |
| Azure AD / Entra ID | Yes |
| Okta | Yes |
| Google | Yes |
| GitHub | Yes |
| LDAP | Yes |
SSO groups can be mapped to teams with a default access level (basic/plus/admin), enabling automatic user provisioning when they authenticate.
### Feature Gates
Controlled feature releases use a `FeatureGate` system with:
* Release states: `internal_testing` → `enterprise_beta` → `public`
* Visibility flags per audience (superadmin, enterprise, public)
* API endpoint mapping for backend enforcement
* Frontend `useFeatureGate()` hook for conditional UI rendering
### Summary
| Capability | Status |
| ---------------------------------------- | -------------------------------------------------------------------------------------------------- |
| Basic roles (Admin, User/Member, Viewer) | **Yes** — Plus Owner, Guest, and Billing Manager |
| Custom roles and permissions | **Yes** — Granular `resource:action` permissions assignable to any role at org/team/resource scope |
| Resource-level access control | **Yes** — Per-dataset and per-model grants with time/token/rate restrictions |
| Role hierarchy | **Yes** — Organization → Team → Resource with cascading permissions |
| Audit logging | **Yes** — Full action log with actor, target, and context |
| SSO auto-provisioning | **Yes** — Azure AD, Okta, Google, GitHub, LDAP |
| Feature gating | **Yes** — Phased rollout with permission-based visibility |
## Security best practices
To maximize security when using 4MINDS:
* Rotate API keys regularly
* Use OAuth when possible instead of password authentication
* Monitor your notification alerts for unusual activity
* Review connected integrations periodically and disconnect unused sources
* Keep your authentication credentials private and never share API keys
Security is a shared responsibility. We provide the infrastructure and controls, and you maintain secure practices when accessing and using the platform.
# SYMI Agentic AI Platform
Source: https://docs.4minds.ai/symi
SYMI is 4MINDS' agentic AI layer — an autonomous workflow engine that sits on top of your trained models and acts on your behalf across email, CRM, and other connected systems.
**SYMI is currently in beta.** Features, behavior, and configuration options may change as we continue to refine the platform. We recommend testing SYMI workflows in non-production environments and sharing feedback with your 4MINDS contact.
## Overview
![Screenshot 2026 05 28 At 2 44 13 PM]()
SYMI is a separate product layer built on top of the 4MINDS model platform. While the model platform handles knowledge ingestion, training, and inference, SYMI handles **action** — triggering automatically on incoming events, routing requests across your models, running scheduled jobs, and pushing outputs to external systems without manual intervention.
SYMI runs on its own base model (Qwen 3.6) and can route requests to any 4MINDS model you have trained. It is designed for teams that want to automate multi-step workflows — not just answer questions, but take action based on what it finds.
SYMI is a separate product layer. You must have at least one trained 4MINDS model before configuring SYMI workflows. SYMI can query your models but cannot replace them.
## Key components
SYMI is composed of six core components that work together to power agentic workflows:
| Component | Description |
| --------------------------------- | ----------------------------------------------------------------------------------------- |
| **Model Router** | Selects which 4MINDS model SYMI queries for a given task |
| **Connections** | Integrations with external systems (Gmail, Outlook, HubSpot, Slack) |
| **Chat Window** | Interactive interface for testing and running SYMI sessions manually |
| **Sub-Agents** | Specialized agents scoped to specific tasks, each with their own prompt and model routing |
| **Scheduler** | Cron-style recurring job runner — trigger workflows on a set cadence |
| **Reasoning Log (Constellation)** | Audit trail of SYMI's thinking steps across a session |
## Model Router
The Model Router controls which 4MINDS model SYMI uses to answer a query or complete a task. By default, SYMI uses its own Qwen 3.6 base model. You can configure the router to direct requests to any trained model in your workspace.
SYMI can intelligently query **across multiple models** when instructed — for example, routing a security question to your SOC model while routing a product question to your general knowledge model.
**To configure the Model Router:**
1. Open SYMI from the main navigation.
2. Navigate to **Model Router** settings.
3. Select the default model for general queries.
4. Configure per-sub-agent model routing for specialized workflows.
## Connections
Connections link SYMI to external systems so it can receive triggers, read data, and push outputs.
### Supported connections
| Connection | Capabilities | Auto-trigger on inbound? |
| ------------------------ | -------------------------------------------------- | ------------------------ |
| **Gmail** | Send + receive email; auto-trigger on inbound mail | ✅ Yes |
| **Outlook / Office 365** | Send + receive email; scheduled poll workaround | ❌ No (see note below) |
| **HubSpot** | Read and write CRM records | N/A |
| **Slack** | Send messages to channels | N/A |
**Gmail is currently the only connection that supports automatic session triggering on inbound email.** When a new email arrives in a Gmail inbox connected to SYMI, it can instantly trigger a workflow — no manual action needed.
Outlook and Office 365 do not support inbound auto-trigger at this time. Teams using enterprise email via Outlook can work around this by configuring the Scheduler to poll the inbox at regular intervals (e.g. every 15 minutes).
### Setting up a Gmail connection
Gmail auto-trigger is the recommended approach for event-driven agentic workflows. Example use case: a ServiceNow ticket is created → an automated email is sent to a SYMI-monitored Gmail inbox → SYMI reads the email, queries your knowledge graph, and responds with recommended next steps.
**Steps:**
1. In SYMI, navigate to **Connections**.
2. Select **Gmail** and authenticate with the account you want SYMI to monitor.
3. Configure the trigger condition (e.g. all inbound mail, or filtered by subject/sender).
4. Link the trigger to a sub-agent or workflow.
Use a dedicated Gmail account for SYMI integrations — do not connect a personal or primary production inbox.
## Sub-agents
Sub-agents are specialized agents you deploy within SYMI for specific tasks. Each sub-agent has:
* A **label** — a name identifying its purpose (e.g. "RFP Responder", "SOC Triage Agent").
* A **task prompt** — the instructions that define what the sub-agent does and how it should behave.
* A **model routing assignment** — which 4MINDS model the sub-agent queries.
Sub-agent task prompts are where you define role-specific context for SYMI — for example, *"You are a cybersecurity analyst working in a SOC. When given an alert, identify the affected system, classify the severity, and suggest a remediation step."* This is different from Personas, which control tone only. For system-level role definitions, always use the sub-agent task prompt.
**To create a sub-agent:**
1. In SYMI, navigate to **Sub-Agents**.
2. Click **+ New Sub-Agent**.
3. Enter a label, task prompt, and select the model to route to.
4. Save and activate.
### Example: RFP response workflow
A pre-built example of what a sub-agent workflow looks like end-to-end:
1. SYMI monitors a Gmail inbox for emails with **"RFP"** in the subject line.
2. On trigger, the sub-agent extracts the scope and requirements from the email body.
3. SYMI queries the connected 4MINDS model (trained on your proposals and org knowledge) to generate a draft response.
4. The draft is pushed to **HubSpot** for internal review and approval before any external send.
## Scheduler
The Scheduler lets you run SYMI workflows on a recurring cadence without needing an inbound trigger. Jobs are configured using cron-style timing — hourly, daily, weekly, or a custom interval.
**Example use cases:**
* Run a threat intelligence summary every morning at 7 AM using your SOC model.
* Poll an S3 bucket every few hours for new log events and produce an incident report.
* Send a weekly digest of new HubSpot activity to a Slack channel.
**To configure a scheduled job:**
1. In SYMI, navigate to **Scheduler**.
2. Click **+ New Job**.
3. Define the task (either a sub-agent or a direct prompt).
4. Set the schedule interval.
5. Activate the job.
## Reasoning Log (Constellation)
Every SYMI session generates a **Reasoning Log** — an audit trail of the steps SYMI took to arrive at its output. This is powered by Constellation, 4MINDS' structured cognitive reasoning system.
The Reasoning Log shows a **summary-level view** of SYMI's thinking steps. It does not expose the full context window.
**What the Reasoning Log captures:**
* Which models were queried.
* What sub-agents were invoked.
* Key reasoning steps and decision points.
* What was sent to external systems.
This log is useful for debugging workflows, verifying that the right model was used, and demonstrating reasoning transparency to stakeholders.
## Slash commands
SYMI supports a library of slash commands in the Chat Window for quick access to platform controls during a session. The slash command toolkit covers:
| Category | Examples |
| ------------------------ | ----------------------------------------- |
| **Session management** | Start, end, or reset a session |
| **Model routing** | Switch the active model mid-session |
| **Reasoning control** | Toggle reasoning depth or transparency |
| **Tool & plugin access** | Invoke specific tools or integrations |
| **Agent deployment** | Launch or configure sub-agents inline |
| **Integrations** | Manage connection status during a session |
Type `/` in the SYMI Chat Window to browse available commands.
## Example architecture: email-triggered agentic workflow
The following illustrates a complete event-driven workflow using SYMI's Gmail auto-trigger:
```text theme={null}
Inbound email → Gmail inbox (monitored by SYMI)
↓
SYMI triggers automatically
↓
Sub-agent reads email, extracts key information
↓
Model Router queries relevant 4MINDS model (e.g. SOC model, General KB)
↓
SYMI generates response or report
↓
Output pushed to HubSpot / Slack / email reply
```
This workflow runs entirely without human intervention once configured.
## Frequently asked questions
**Can SYMI query multiple models in a single session?**
Yes. You can configure sub-agents to route to different models, allowing a single SYMI workflow to pull from multiple knowledge sources — for example, your SOC model for security context and your General KB model for company policy.
**Can I use SYMI without a Gmail account?**
Yes, but auto-triggering on inbound email requires Gmail at this time. Outlook users can configure the Scheduler to poll their inbox at regular intervals as a workaround. Broader auto-trigger support for Outlook and other providers is on the roadmap.
**Is SYMI the right place to define role-specific context for my AI?**
Yes — for system-level role definitions like *"You are a cybersecurity expert in a SOC"*, use the **sub-agent task prompt** in SYMI. The **Persona** feature at the model level controls tone and communication style only, not role context.
**Does SYMI have access to my knowledge graph directly?**
SYMI queries your 4MINDS models, and those models are connected to your knowledge graph. SYMI does not access the Synthesis Graph™ directly — all knowledge retrieval happens through model inference endpoints.
## Next steps
Connect external data sources to the models SYMI will query
View and manage the models available for SYMI to route to
Understand the difference between Personas and SYMI sub-agent task prompts
Explore definitions for SYMI, sub-agents, and agentic workflows
# Organize your Teams
Source: https://docs.4minds.ai/teams
The Teams section is your central hub for managing your organization, its members, and shared resources within 4minds.ai. It provides a unified view of organizational settings, team structure, and any models or datasets shared across your workspace. To access Teams, click your profile icon in the top-right corner of the screen and select Teams from the dropdown menu.
![Screenshot2026 03 09at6 03 09PM]()
![Screenshot2026 03 09at6 03 09PM]()
**Availability:** Teams functionality is available on all paid plans. Seat limits and advanced features (SSO, audit logs) vary by tier.
| Tier | Max Team Members | SSO | Audit Logs |
| :------------- | :--------------- | :-- | :--------- |
| **Pro** | 5 | No | No |
| **Teams** | 20 | Yes | Yes |
| **Teams Plus** | 50 | Yes | Yes |
| **Enterprise** | Unlimited | Yes | Yes |
***
## Roles & Access Levels
### Organization Roles
| Role | Description |
| :------------------ | :------------------------------------------------------------------------------------------------------------------------------ |
| **Owner** | Full permissions including org deletion, billing, and all team/member management. Cannot be removed or have their role changed. |
| **Admin** | Can manage teams, members, and org settings. |
| **Billing Manager** | Handles billing operations for the organization. |
| **Member** | Standard access within the organization. |
| **Guest** | Limited, restricted access. |
### Access Levels
Access levels control what users can do with resources across the platform. They appear throughout the UI and are assigned per member.
| Access Level | Capabilities |
| :----------- | :--------------------------------------------------------------- |
| **Basic** | Read-only access to datasets and models. |
| **Plus** | Everything in Basic, plus create and update datasets/models. |
| **Admin** | Full access including delete, team creation, and administration. |
> **Note:** Access levels map to subscription tiers - Pro/Teams/Teams Plus users receive Plus access; Enterprise users receive Admin access by default.
***
## Organization Panel
At the top of the page, your **Organization** card displays:
* **Organization name**
* **Your role** (e.g., Owner)
* **Current subscription tier** (e.g., Enterprise)
* A summary of **total members** and **total teams**
***
## Teams List
The main area of the Teams tab displays all teams within your organization as **Team Cards**. Each card shows:
* Team **name** and **description**
* **Member count**
* Your **role badge** within that team
* Linked **SSO providers** (if configured)
* **Public/Private** visibility status
### Creating a Team
Click **+ Create New Team** from the Quick Actions panel. The creation modal includes:
* **Team Name** (required)
* **SSO Auto-Membership** configuration (optional) — configure during creation to automatically provision members from an SSO group:
* SSO Provider
* Identifier type
* Group identifier
* Default permission level for auto-provisioned members
### Nested Teams
Teams support a **parent-child hierarchy**. When creating or editing a team, you can optionally assign a parent team to build structured team trees within your organization.
### Public vs. Private Teams
Each team has a **visibility flag**:
* **Public** — Visible to all organization members
* **Private** — Only visible to team members and admins
***
## Team Detail View
Clicking into a team opens its **detail view**, which includes a full members table with the following columns:
| Column | Description |
| :------------------ | :------------------------------------------------------- |
| **Name & Username** | Member's display name and username |
| **Email** | Member's email address |
| **Joined** | Date the member joined the team |
| **Role** | Member's role within the team |
| **Access Level** | Basic, Plus, or Admin |
| **Source** | How the member was added — SSO provider name or "Manual" |
| **Options** | Actions available for that member |
### Member Management
* Members must be **organization members first** before being added to a team
* When adding a member, specify their **access level** (Basic / Plus / Admin)
* Member permissions can be **updated after joining** via the permissions panel
* Use the **search/filter** bar to find members within large teams
* **Bulk operations** — select multiple members and bulk remove them. Removal requires typing `REMOVE` to confirm
> **Note:** The organization owner cannot be removed or have their role changed.
***
## Team Settings Panel
Each team has a dedicated settings panel (accessible from the team detail view) displaying:
* **Stats:** Members, Admins, Pending Invites
* **Editable team name**
* **SSO Auto-Membership mappings** — configure which SSO groups automatically provision members into this team:
| Provider | Supported Identifier Types |
| :------------------- | :------------------------- |
| **Azure Entra ID** | object\_id, name, email |
| **Okta** | name, email |
| **Google Workspace** | email |
| **GitHub** | team\_slug |
| **LDAP** | DN |
Each SSO mapping includes a **default permission level** for auto-provisioned members.
***
## Invitation System
Admins can invite new users to the organization. Invitations support:
* **Statuses:** Pending, Accepted, Declined, Expired, Revoked
* **Auto-join:** Specify a team and access level so the user is automatically placed on acceptance
* **Seat limit enforcement** based on your subscription tier
* **Privacy-conscious design** — the system does not reveal whether an email address already has an account
***
## Shared Resources
The **Shared Resources** section displays all models and datasets shared with you by other organization members. It is divided into two sub-tabs:
### Shared Models
Displays a table of all models shared with you:
| Column | Description |
| :------------- | :------------------------------------------------- |
| **Name** | The name of the shared model |
| **Parameters** | Configuration parameters associated with the model |
| **Base** | The base model it was built upon |
| **Persona** | The persona assigned to the model |
| **Created** | The date the model was created |
| **Actions** | Available actions you can perform on the model |
> If no models have been shared yet, a **"No Shared Models"** placeholder will be displayed.
### Shared Datasets
Displays datasets shared with you by other team members. A dataset can be shared with the entire organization, specific teams, or individual users.
***
## Organization Settings Panel (Right Sidebar)
The right-hand sidebar provides quick access to your organization's configuration and key stats.
### Summary Stats
| Stat | Description |
| :------------------ | :---------------------------------------------------- |
| **Members** | Total number of members in the organization |
| **Teams** | Total number of teams created |
| **Admins** | Number of users with admin privileges |
| **Pending Invites** | Number of outstanding invitations awaiting acceptance |
### General Settings
* **Org Name** - View or update the display name of your organization.
* **Current Tier** - Displays the active subscription plan. Read-only.
* **Allowed Email Domains** - Define which email domains are permitted to log in via SSO and be auto-matched to your organization. Add new domains via the input field and **+ Add** button, or remove existing ones using the delete icon.
### AD/LDAP Integration
The **AD/LDAP Integration** tab allows you to connect [4minds.ai](http://4minds.ai) to your directory service for centralized authentication.
**Azure AD mode** requires:
* Tenant ID, Client ID, Client Secret
* Enable/disable toggle
**LDAP mode** requires:
* Server URL, Bind DN, Bind Password
* Base DN, User Search Filter, Group Attribute
* SSL/TLS and STARTTLS options
* Certificate validation toggle
Both modes support:
* **Test Connection** button with live status feedback
* **AD Preview** showing mapped groups
* **Auto-team creation** from AD/LDAP groups
***
## Quick Actions
Located at the bottom of the right sidebar:
* **+ Create New Team** — Set up a new team within your organization.
* **+ Add Member(s)** — Invite new users to join your organization.
***
## Audit Trail
All RBAC actions - role changes, permission grants, member additions and removals - are logged in a full **audit trail** with actor, target, resource, IP address, and timestamp. Audit logs are available on **Teams tier and above**.
***
## Organization Deletion
Only the **Owner** can delete an organization. Deletion requires a confirmation step. Upon deletion, resources (models, datasets, etc.) are **orphaned but not deleted**.
***
## Tips
> **Access Levels are core:** Basic, Plus, and Admin access levels appear throughout the UI. Make sure members are assigned the right level when they join a team.
> **SSO Auto-Membership saves time:** Configure SSO group mappings on your teams so new hires are automatically provisioned to the right teams on first login - no manual steps required.
> **Use nested teams for large orgs:** Parent-child team hierarchies help organize departments and sub-teams without losing visibility across the org.
## Organization Panel
At the top of the page, you will see your **Organization** card displaying:
* **Organization name** (e.g., [4minds.ai](http://4minds.ai))
* **Your role** within the organization (e.g., Owner)
* **Current subscription tier** (e.g., Enterprise)
* A summary of **total members** and **total teams** belonging to the organization
***
## Shared Resources
The **Shared Resources** section allows you to view all models and datasets that other team members have shared with you. It is divided into two sub-tabs:
### Shared Models
Displays a table of all models shared with you, including the following columns:
| Column | Description |
| :------------- | :------------------------------------------------- |
| **Name** | The name of the shared model |
| **Parameters** | Configuration parameters associated with the model |
| **Base** | The base model it was built upon |
| **Persona** | The persona assigned to the model |
| **Created** | The date the model was created |
| **Actions** | Available actions you can perform on the model |
> If no models have been shared yet, a **"No Shared Models"** placeholder will be displayed.
### Shared Datasets
Displays datasets shared with you by other team members. Switching to this sub-tab will list all available shared datasets in your organization.
***
## Organization Settings Panel (Right Sidebar)
The right-hand sidebar provides quick access to your organization's configuration and statistics.
### Summary Stats
| Stat | Description |
| :------------------ | :---------------------------------------------------- |
| **Members** | Total number of members in the organization |
| **Teams** | Total number of teams created |
| **Admins** | Number of users with admin privileges |
| **Pending Invites** | Number of outstanding invitations awaiting acceptance |
### General Settings
* **Org Name** – View or update the display name of your organization.
* **Current Tier** – Displays the active subscription plan (e.g., Enterprise). This is read-only.
* **Allowed Email Domains** – Define which email domains are permitted to log in via SSO and be auto-matched to your organization. You can add new domains using the input field and the **+ Add** button, or remove existing ones using the delete icon.
### AD/LDAP Integration Tab
Switching to this tab allows you to configure Active Directory or LDAP-based authentication for your organization.
***
## Quick Actions
Located at the bottom of the right sidebar, Quick Actions give you fast access to common team management tasks:
* **+ Create New Team** – Set up a new team within your organization.
* **+ Add Member(s)** – Invite new users to join your organization.
***
## Tips
> **Note:** Only users with **Owner** or **Admin** roles can modify Org Settings and manage members.
> **SSO Tip:** Domains added under **Allowed Email Domains** enable seamless SSO onboarding — users with matching email addresses will be automatically associated with your organization upon login.