- AWS-level authentication — how 4MINDS discovers which Redshift clusters and Serverless workgroups exist in your account. Uses IAM Role Federation (recommended) or Amazon Cognito, both shared with the rest of 4MINDS’s AWS integrations.
- Database-level authentication — uses temporary Redshift credentials minted from the same AWS credentials used for discovery. Provisioned clusters use
redshift:GetClusterCredentials; Serverless workgroups useredshift-serverless:GetCredentials. No long-lived database password is stored.
Provisioned vs. Serverless
| Provisioned cluster | Redshift Serverless | |
|---|---|---|
| Discovery API | redshift:DescribeClusters | redshift-serverless:ListWorkgroups |
| Credential API | redshift:GetClusterCredentials | redshift-serverless:GetCredentials |
| Database user | You choose the DB user (see Database User) | Derived automatically from the IAM identity — no DB user to configure |
| Default port | 5439 | 5439 |
| Shown in picker as | Node type + node count | Serverless · <namespace> |
Redshift IAM Permissions Policy
Whichever AWS auth method you use (IAM Role Federation or Cognito), attach this policy to the role. It covers both provisioned and Serverless. If you only use one, you can drop the statements for the other.| Permission | Purpose |
|---|---|
redshift:DescribeClusters | Lists provisioned clusters in the connected region so 4MINDS can populate the picker (also backs Test Connection) |
redshift:GetClusterCredentials | Mints short-lived DB credentials for a provisioned cluster + DB user |
redshift-serverless:ListWorkgroups | Lists Serverless workgroups in the connected region |
redshift-serverless:GetCredentials | Mints short-lived DB credentials for a Serverless workgroup |
4MINDS-Redshift-Access — you’ll attach it when creating the IAM role (for Role Federation) or the Cognito authenticated role.
Note:GetClusterCredentialsis authorized against both thedbuser:anddbname:resource ARNs. The wildcards above allow all clusters, DB users, and databases. To restrict, narrow the ARNs (e.g.,arn:aws:redshift:us-east-1:123456789012:dbuser:my-cluster/fourminds_iam_user).
Least-Privilege: Restricting to Specific Resources
To expose only a subset of your Redshift resources, scope the discovery and credential ARNs:Database Access & Grants
The IAM identity 4MINDS assumes must map to a database user that has read access to the tables you want to import.Provisioned clusters — the DB user
For a provisioned cluster, 4MINDS callsGetClusterCredentials with a DB username you configure (see Database User). That user must already exist in the cluster and have SELECT on the target tables. AutoCreate is not used — 4MINDS never creates users in your cluster.
dbuser: ARN must match this username (e.g., .../my-cluster/fourminds_iam_user).
Serverless workgroups — IAM-derived user
Redshift ServerlessGetCredentials does not accept a DB username — it derives the database user from the IAM identity making the call (returned as IAMR:<role-name> or IAM:<user-name>). To grant it access, create/grant that exact user in the database:
Tip: If you’re unsure of the exact derived username, run a browse in 4MINDS and check the error message — apermission deniedfailure will name the DB user Redshift resolved, which you can thenGRANTto.
Note: For Serverless,GetCredentialsauto-creates theIAMR:<role>database user on first use, so the explicitCREATE USERabove is optional — grantingSELECTtoPUBLIC(or to the resolved user after a first browse) is sufficient. Pre-creating the user is only needed if you want to grant privileges before the first connection.
Connection Method 1: IAM Role Federation (Recommended)
Short-lived credentials minted per request through AWS STS — no long-lived AWS keys stored in 4MINDS.- Complete the one-time AWS setup in AWS Integrations → IAM Role Federation (OIDC provider + IAM role with trust policy).
- Attach the Redshift IAM policy above to the role (Role Federation and Cognito can share the same role — just attach both service policies to it if you use multiple AWS integrations).
- In 4MINDS, open Integrations → Amazon Redshift → click Configure (or Settings if already connected).
- Keep the IAM Role tab selected.
- Paste your IAM Role ARN (e.g.,
arn:aws:iam::123456789012:role/4MINDS-integration-role). - Leave the External ID field blank — it is not supported for IAM Role Federation (see AWS Integrations → Verify the Trust Policy).
- Enter your AWS Region (e.g.,
us-east-1) — this must match the region of the Redshift resources you want to browse. - (Provisioned only) Enter a Database user — see Database User. Leave blank to use the default
fourminds_iam_user. - Click Test Connection — success shows “Connection successful. Found N Redshift cluster(s).”
- Click Save Credentials.
Connection Method 2: Amazon Cognito
Use this if your organization already authenticates against AWS through Cognito User Pools and Identity Pools.- Complete the one-time Cognito setup in AWS Integrations → Amazon Cognito (User Pool, App Client, Identity Pool, authenticated role).
- Attach the Redshift IAM policy above to the Cognito authenticated role.
- In 4MINDS, open Integrations → Amazon Redshift → click Configure.
- Switch to the Cognito tab.
- Fill in User Pool ID, App Client ID, App Client Secret (if configured), Identity Pool ID, Username, and Password.
- Enter your AWS Region.
- (Provisioned only) Enter a Database user if you’re not using the default.
- Click Test Connection, then Save Credentials.
Database User
GetClusterCredentials (provisioned clusters only) authenticates as a specific database user. In the settings modal, the Database user field controls which user 4MINDS requests credentials for:
- Defaults to
fourminds_iam_userif left blank. - The user must already exist in the cluster with
SELECTon the tables you want to import (see Database Access & Grants). - The value is stored once with your connection metadata and reused for every browse and import — you don’t re-enter it per request.
- Ignored for Serverless — Serverless derives the user from the IAM identity, so this field has no effect on workgroups.
4MINDS Fields
| Field | Required | Notes |
|---|---|---|
| AWS Region | Yes | Region where your Redshift resources are deployed. Must match the region of your Cognito resources (Cognito method) |
| IAM Role ARN | IAM Role method | Role with the Redshift IAM policy attached |
| External ID | No | Not supported for IAM Role Federation — leave blank |
| Database user | No | Provisioned clusters only. Defaults to fourminds_iam_user. Ignored for Serverless |
| Cognito fields | Cognito method | See AWS Integrations |
Importing Data from Redshift
Once connected, open the Datasets page and click Add Source → Amazon Redshift (or pick Redshift from the data source bar inside an existing dataset). The import wizard walks you through the hierarchy: clusters/workgroups → databases → tables.Step 1 — Pick a Cluster or Workgroup
4MINDS callsredshift:DescribeClusters and redshift-serverless:ListWorkgroups in your configured region and shows everything the role can see. Provisioned clusters show their node type and node count; Serverless workgroups are labeled Serverless with their namespace. Click one to continue.
If the Serverless permissions are missing, workgroups are simply omitted — provisioned clusters still list normally.
Step 2 — Browse Databases
4MINDS mints a short-lived credential (viaGetClusterCredentials for a cluster, or GetCredentials for a workgroup), opens a live SQL connection to the endpoint over SSL, and lists user databases. System databases (template0, template1, padb_harvest, rdsadmin) are filtered out.
Step 3 — Select Tables
Click a database to list its base tables. For each table the browser shows:- Schema-qualified name (e.g.,
public.customers) - Column count
- On-disk size (when available)
During Import
At import time, 4MINDS runs a bulk fetch on each selected table:- Up to 50,000 rows per table (hard cap — larger tables are truncated and flagged).
- Binary columns (
BYTEA,VARBINARY,BINARY) are dropped from the export rather than base64-encoded. - Complex types (arrays, structured values) are serialized with
json.dumpsinto a single cell. - Timeout is 120 seconds per table.
Dataset Sync
Overview
Beyond one-time imports, 4MINDS can keep a Redshift-backed dataset up to date automatically. When you enable Dataset Sync on a dataset built from Redshift tables, 4MINDS re-exports the configured tables on a schedule — no manual re-imports. Unlike file-based sources (S3, Dropbox, Databricks volumes), Redshift is a table warehouse with no cheap per-table “last modified” signal. So Redshift sync follows the same full-snapshot model as BigQuery rather than incremental file diffing: each sync cycle re-exports the whole table (up to the 50,000-row cap) and replaces the previous snapshot. There is no row-level change detection — every run is a fresh point-in-time copy.How to Set Up Sync
- Import one or more Redshift tables into a 4MINDS dataset (using the import wizard above).
- On the dataset, toggle Dataset Sync on.
- Select a sync frequency (see table below).
- From that point on, 4MINDS re-exports each configured table at the chosen interval.
Sync Frequencies
| Frequency | Interval | Best For |
|---|---|---|
| Every minute | 1 minute | Rapidly changing tables (Enterprise tier) |
| Hourly | 1 hour | Frequently updated warehouse tables (Teams & Enterprise) |
| Daily | 24 hours | Standard business reporting (All paid tiers) |
| Weekly | 7 days | Slowly changing reference data |
| Monthly | 30 days | Compliance snapshots, archival data |
Because every Redshift sync is a full-table re-export, prefer less frequent intervals for large tables to control export cost and RPU/credit usage. Daily or weekly is a good default for warehouse tables.
How It Works Internally
Full snapshot, every cycle: Because Redshift exposes no per-table modification timestamp, the sync manifest records nomodified_at for Redshift entries. As a result, the writer treats every configured table as changed on every run and re-downloads a fresh snapshot — the same semantic BigQuery uses.
Stable table identity: Each table is tracked by a stable identifier of the form rs-<cluster>-<database>-<schema>-<table> (schema defaults to public). The manifest de-duplicates on this id, so re-running a sync replaces the prior snapshot of a table rather than accumulating duplicates.
Export pipeline: On each cycle, for every configured table 4MINDS mints a short-lived Redshift credential (via GetClusterCredentials for a cluster, or GetCredentials for a workgroup), opens an SSL SQL connection, and exports the table to CSV bytes — up to 50,000 rows (the same cap as one-shot import; larger tables are truncated and flagged). Binary columns are dropped and empty tables are skipped. The CSV is uploaded to the user’s dataset storage and processed through the 4MINDS ETL pipeline.
Concurrent processing: As with other sources, multiple dataset syncs run in parallel, each with its own isolated database session.
Authentication for Automated Sync
Dataset sync runs in the background with no user present, so 4MINDS replays the stored AWS connection to authenticate each cycle — exactly the same two-layer model used for interactive browsing:- The stored auth method (IAM Role Federation or Cognito) is resolved into fresh temporary AWS credentials for the run (STS
AssumeRoleWithWebIdentity, or CognitoGetCredentialsForIdentity). - Those credentials mint a fresh temporary Redshift credential per table (
GetClusterCredentials/GetCredentials).
SELECT on the synced tables (see Database Access & Grants) — a revoked grant surfaces as a skipped table in the sync logs.
Networking Requirements
Redshift clusters and Serverless workgroups live inside a VPC. Only the 4MINDS backend opens SQL connections to Redshift — the browser app (app.4minds.ai) never connects to your database directly, so you only need to allow the backend’s egress IP. There is no frontend IP to allowlist.
For 4MINDS to open a SQL connection, the endpoint must be reachable from the backend’s outbound network:
-
Allow the 4MINDS backend egress IP on the Redshift port (default
5439):Add it as an inbound rule on the security group attached to your cluster or Serverless workgroup. AWS CLI:Environment Backend hostname Egress IP to allowlist Production api.4minds.ai20.7.240.218/32Or in the console: EC2 → Security Groups → your Redshift SG → Inbound rules → Edit inbound rules → Add rule, then set Type =Custom TCP, Port range =5439, Source =20.7.240.218/32. - Publicly accessible = Yes is the simplest path for temporary testing; for production, prefer a VPC peering / PrivateLink setup with 4MINDS support.
-
SSL — All connections use SSL (
sslmode=require), as required for temporary-credential auth.
Note: If a connection still times out after allowlisting the IP above, the backend egress IP may have changed — contact 4MINDS support to confirm the current address before widening your rules.If the SQL connection times out, the browser shows:
“Connection timed out. The backend may not be able to reach the Redshift endpoint — check VPC / security group rules (inbound port open from the backend’s network).”
Security Model
| Credential | Where it lives | How long |
|---|---|---|
| AWS IAM Role ARN / Cognito config | 4MINDS database (encrypted metadata) | Until you disconnect |
| Temporary AWS credentials (STS) | Request memory only | ~1 hour, minted per call |
Redshift DB credentials (Get(Cluster)Credentials) | Request memory only | ~1 hour, generated per call |
Testing Your Connection
After saving credentials, the Test Connection button:- Resolves your auth method → AWS credentials (via STS AssumeRoleWithWebIdentity for Role Federation, or Cognito GetCredentialsForIdentity for Cognito).
- Calls
redshift:DescribeClusterswith a 15-second timeout. - Returns “Connection successful. Found N Redshift cluster(s).” on success.
Troubleshooting
| Issue | Solution |
|---|---|
AccessDenied on Test Connection | The IAM policy isn’t attached to the role, or is missing redshift:DescribeClusters |
| Serverless workgroups don’t appear | The role is missing redshift-serverless:ListWorkgroups, or there are no workgroups in that region |
Invalid Access Key ID / Invalid Secret Access Key | Wrong static credentials (only relevant if using legacy access-keys auth) |
Unrecognized client | Wrong AWS region, or Redshift isn’t enabled in that region |
Could not resolve Redshift endpoint hostname | The endpoint in the list is wrong, or DNS can’t resolve it from the backend |
Connection timed out (SQL step) | Security group doesn’t allow inbound port 5439 from the 4MINDS backend egress IP (20.7.240.218/32) — see Networking Requirements |
Connection refused | Port 5439 isn’t open on the security group, or the endpoint isn’t reachable and there’s no peering |
Authentication failed | The IAM policy is missing redshift:GetClusterCredentials / redshift-serverless:GetCredentials, or the resolved DB user lacks a login/grant. See Database Access & Grants |
permission denied for relation ... | The DB user exists but lacks SELECT on the table — run the GRANT statements in Database Access & Grants |
SSL error connecting to Redshift | Temporary-credential auth requires SSL — AWS-managed certificates work out of the box |
| Tables list is empty | The database has no base tables, or the DB user lacks access to information_schema |
Disconnecting
To remove the Redshift integration:- Open Integrations → Amazon Redshift → click Settings.
- Click Disconnect.