Skip to main content
This guide walks you through connecting 4MINDS to Amazon Redshift. 4MINDS supports both provisioned clusters and Redshift Serverless workgroups — they appear together in the same picker. The connection has two layers:
  1. AWS-level authentication — how 4MINDS discovers which Redshift clusters and Serverless workgroups exist in your account. Uses IAM Role Federation (recommended) or Amazon Cognito, both shared with the rest of 4MINDS’s AWS integrations.
  2. Database-level authentication — uses temporary Redshift credentials minted from the same AWS credentials used for discovery. Provisioned clusters use redshift:GetClusterCredentials; Serverless workgroups use redshift-serverless:GetCredentials. No long-lived database password is stored.
This two-layer model lets 4MINDS list your Redshift resources and connect to individual databases with a single set of AWS credentials — no long-lived database passwords to manage or rotate. For the generic IAM Role / Cognito setup (creating the OIDC provider, trust policy, Cognito user pool, etc.), see AWS Integrations. The steps below focus on what’s specific to Redshift.

Provisioned vs. Serverless

Provisioned clusterRedshift Serverless
Discovery APIredshift:DescribeClustersredshift-serverless:ListWorkgroups
Credential APIredshift:GetClusterCredentialsredshift-serverless:GetCredentials
Database userYou choose the DB user (see Database User)Derived automatically from the IAM identity — no DB user to configure
Default port54395439
Shown in picker asNode type + node countServerless · <namespace>
Both types are listed together when you browse Redshift in the import wizard. The difference is handled server-side; you don’t have to pick a mode.

Redshift IAM Permissions Policy

Whichever AWS auth method you use (IAM Role Federation or Cognito), attach this policy to the role. It covers both provisioned and Serverless. If you only use one, you can drop the statements for the other.
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "RedshiftProvisionedDiscovery",
      "Effect": "Allow",
      "Action": [
        "redshift:DescribeClusters"
      ],
      "Resource": "*"
    },
    {
      "Sid": "RedshiftProvisionedCredentials",
      "Effect": "Allow",
      "Action": "redshift:GetClusterCredentials",
      "Resource": [
        "arn:aws:redshift:*:*:dbuser:*/*",
        "arn:aws:redshift:*:*:dbname:*/*"
      ]
    },
    {
      "Sid": "RedshiftServerlessDiscovery",
      "Effect": "Allow",
      "Action": [
        "redshift-serverless:ListWorkgroups"
      ],
      "Resource": "*"
    },
    {
      "Sid": "RedshiftServerlessCredentials",
      "Effect": "Allow",
      "Action": "redshift-serverless:GetCredentials",
      "Resource": "*"
    }
  ]
}
PermissionPurpose
redshift:DescribeClustersLists provisioned clusters in the connected region so 4MINDS can populate the picker (also backs Test Connection)
redshift:GetClusterCredentialsMints short-lived DB credentials for a provisioned cluster + DB user
redshift-serverless:ListWorkgroupsLists Serverless workgroups in the connected region
redshift-serverless:GetCredentialsMints short-lived DB credentials for a Serverless workgroup
Name the policy something memorable like 4MINDS-Redshift-Access — you’ll attach it when creating the IAM role (for Role Federation) or the Cognito authenticated role.
Note: GetClusterCredentials is authorized against both the dbuser: and dbname: resource ARNs. The wildcards above allow all clusters, DB users, and databases. To restrict, narrow the ARNs (e.g., arn:aws:redshift:us-east-1:123456789012:dbuser:my-cluster/fourminds_iam_user).

Least-Privilege: Restricting to Specific Resources

To expose only a subset of your Redshift resources, scope the discovery and credential ARNs:
{
  "Effect": "Allow",
  "Action": "redshift:GetClusterCredentials",
  "Resource": [
    "arn:aws:redshift:us-east-1:123456789012:dbuser:prod-analytics/fourminds_iam_user",
    "arn:aws:redshift:us-east-1:123456789012:dbname:prod-analytics/dev"
  ]
},
{
  "Effect": "Allow",
  "Action": "redshift-serverless:GetCredentials",
  "Resource": "arn:aws:redshift-serverless:us-east-1:123456789012:workgroup/*"
}

Database Access & Grants

The IAM identity 4MINDS assumes must map to a database user that has read access to the tables you want to import.

Provisioned clusters — the DB user

For a provisioned cluster, 4MINDS calls GetClusterCredentials with a DB username you configure (see Database User). That user must already exist in the cluster and have SELECT on the target tables. AutoCreate is not used — 4MINDS never creates users in your cluster.
-- run in the target database on your Redshift cluster
CREATE USER fourminds_iam_user PASSWORD DISABLE;
GRANT USAGE ON SCHEMA public TO fourminds_iam_user;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO fourminds_iam_user;
-- so future tables are readable too
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO fourminds_iam_user;
The IAM policy’s dbuser: ARN must match this username (e.g., .../my-cluster/fourminds_iam_user).

Serverless workgroups — IAM-derived user

Redshift Serverless GetCredentials does not accept a DB username — it derives the database user from the IAM identity making the call (returned as IAMR:<role-name> or IAM:<user-name>). To grant it access, create/grant that exact user in the database:
-- <role-name> is the IAM role 4MINDS assumes (Role Federation) or the
-- Cognito authenticated role. Redshift prefixes assumed roles with "IAMR:".
CREATE USER "IAMR:4MINDS-integration-role" PASSWORD DISABLE;
GRANT USAGE ON SCHEMA public TO "IAMR:4MINDS-integration-role";
GRANT SELECT ON ALL TABLES IN SCHEMA public TO "IAMR:4MINDS-integration-role";
ALTER DEFAULT PRIVILEGES IN SCHEMA public
  GRANT SELECT ON TABLES TO "IAMR:4MINDS-integration-role";
Tip: If you’re unsure of the exact derived username, run a browse in 4MINDS and check the error message — a permission denied failure will name the DB user Redshift resolved, which you can then GRANT to.
Note: For Serverless, GetCredentials auto-creates the IAMR:<role> database user on first use, so the explicit CREATE USER above is optional — granting SELECT to PUBLIC (or to the resolved user after a first browse) is sufficient. Pre-creating the user is only needed if you want to grant privileges before the first connection.

Short-lived credentials minted per request through AWS STS — no long-lived AWS keys stored in 4MINDS.
  1. Complete the one-time AWS setup in AWS Integrations → IAM Role Federation (OIDC provider + IAM role with trust policy).
  2. Attach the Redshift IAM policy above to the role (Role Federation and Cognito can share the same role — just attach both service policies to it if you use multiple AWS integrations).
  3. In 4MINDS, open IntegrationsAmazon Redshift → click Configure (or Settings if already connected).
  4. Keep the IAM Role tab selected.
  5. Paste your IAM Role ARN (e.g., arn:aws:iam::123456789012:role/4MINDS-integration-role).
  6. Leave the External ID field blank — it is not supported for IAM Role Federation (see AWS Integrations → Verify the Trust Policy).
  7. Enter your AWS Region (e.g., us-east-1) — this must match the region of the Redshift resources you want to browse.
  8. (Provisioned only) Enter a Database user — see Database User. Leave blank to use the default fourminds_iam_user.
  9. Click Test Connection — success shows “Connection successful. Found N Redshift cluster(s).”
  10. Click Save Credentials.

Connection Method 2: Amazon Cognito

Use this if your organization already authenticates against AWS through Cognito User Pools and Identity Pools.
  1. Complete the one-time Cognito setup in AWS Integrations → Amazon Cognito (User Pool, App Client, Identity Pool, authenticated role).
  2. Attach the Redshift IAM policy above to the Cognito authenticated role.
  3. In 4MINDS, open IntegrationsAmazon Redshift → click Configure.
  4. Switch to the Cognito tab.
  5. Fill in User Pool ID, App Client ID, App Client Secret (if configured), Identity Pool ID, Username, and Password.
  6. Enter your AWS Region.
  7. (Provisioned only) Enter a Database user if you’re not using the default.
  8. Click Test Connection, then Save Credentials.

Database User

GetClusterCredentials (provisioned clusters only) authenticates as a specific database user. In the settings modal, the Database user field controls which user 4MINDS requests credentials for:
  • Defaults to fourminds_iam_user if left blank.
  • The user must already exist in the cluster with SELECT on the tables you want to import (see Database Access & Grants).
  • The value is stored once with your connection metadata and reused for every browse and import — you don’t re-enter it per request.
  • Ignored for Serverless — Serverless derives the user from the IAM identity, so this field has no effect on workgroups.

4MINDS Fields

FieldRequiredNotes
AWS RegionYesRegion where your Redshift resources are deployed. Must match the region of your Cognito resources (Cognito method)
IAM Role ARNIAM Role methodRole with the Redshift IAM policy attached
External IDNoNot supported for IAM Role Federation — leave blank
Database userNoProvisioned clusters only. Defaults to fourminds_iam_user. Ignored for Serverless
Cognito fieldsCognito methodSee AWS Integrations

Importing Data from Redshift

Once connected, open the Datasets page and click Add SourceAmazon Redshift (or pick Redshift from the data source bar inside an existing dataset). The import wizard walks you through the hierarchy: clusters/workgroups → databases → tables.

Step 1 — Pick a Cluster or Workgroup

4MINDS calls redshift:DescribeClusters and redshift-serverless:ListWorkgroups in your configured region and shows everything the role can see. Provisioned clusters show their node type and node count; Serverless workgroups are labeled Serverless with their namespace. Click one to continue.
If the Serverless permissions are missing, workgroups are simply omitted — provisioned clusters still list normally.

Step 2 — Browse Databases

4MINDS mints a short-lived credential (via GetClusterCredentials for a cluster, or GetCredentials for a workgroup), opens a live SQL connection to the endpoint over SSL, and lists user databases. System databases (template0, template1, padb_harvest, rdsadmin) are filtered out.

Step 3 — Select Tables

Click a database to list its base tables. For each table the browser shows:
  • Schema-qualified name (e.g., public.customers)
  • Column count
  • On-disk size (when available)
Check the tables you want to import and click Add Tables. Selected tables are staged into your dataset alongside files from any other source.

During Import

At import time, 4MINDS runs a bulk fetch on each selected table:
  • Up to 50,000 rows per table (hard cap — larger tables are truncated and flagged).
  • Binary columns (BYTEA, VARBINARY, BINARY) are dropped from the export rather than base64-encoded.
  • Complex types (arrays, structured values) are serialized with json.dumps into a single cell.
  • Timeout is 120 seconds per table.
Each table lands in your dataset as a UTF-8 CSV with the original column names as headers.

Dataset Sync

Overview

Beyond one-time imports, 4MINDS can keep a Redshift-backed dataset up to date automatically. When you enable Dataset Sync on a dataset built from Redshift tables, 4MINDS re-exports the configured tables on a schedule — no manual re-imports. Unlike file-based sources (S3, Dropbox, Databricks volumes), Redshift is a table warehouse with no cheap per-table “last modified” signal. So Redshift sync follows the same full-snapshot model as BigQuery rather than incremental file diffing: each sync cycle re-exports the whole table (up to the 50,000-row cap) and replaces the previous snapshot. There is no row-level change detection — every run is a fresh point-in-time copy.

How to Set Up Sync

  1. Import one or more Redshift tables into a 4MINDS dataset (using the import wizard above).
  2. On the dataset, toggle Dataset Sync on.
  3. Select a sync frequency (see table below).
  4. From that point on, 4MINDS re-exports each configured table at the chosen interval.
A single sync configuration can span multiple clusters, workgroups, and databases — each table carries its own connection target (endpoint, port, serverless flag, schema), so tables from different Redshift sources can live in the same synced dataset.

Sync Frequencies

FrequencyIntervalBest For
Every minute1 minuteRapidly changing tables (Enterprise tier)
Hourly1 hourFrequently updated warehouse tables (Teams & Enterprise)
Daily24 hoursStandard business reporting (All paid tiers)
Weekly7 daysSlowly changing reference data
Monthly30 daysCompliance snapshots, archival data
Because every Redshift sync is a full-table re-export, prefer less frequent intervals for large tables to control export cost and RPU/credit usage. Daily or weekly is a good default for warehouse tables.

How It Works Internally

Full snapshot, every cycle: Because Redshift exposes no per-table modification timestamp, the sync manifest records no modified_at for Redshift entries. As a result, the writer treats every configured table as changed on every run and re-downloads a fresh snapshot — the same semantic BigQuery uses. Stable table identity: Each table is tracked by a stable identifier of the form rs-<cluster>-<database>-<schema>-<table> (schema defaults to public). The manifest de-duplicates on this id, so re-running a sync replaces the prior snapshot of a table rather than accumulating duplicates. Export pipeline: On each cycle, for every configured table 4MINDS mints a short-lived Redshift credential (via GetClusterCredentials for a cluster, or GetCredentials for a workgroup), opens an SSL SQL connection, and exports the table to CSV bytes — up to 50,000 rows (the same cap as one-shot import; larger tables are truncated and flagged). Binary columns are dropped and empty tables are skipped. The CSV is uploaded to the user’s dataset storage and processed through the 4MINDS ETL pipeline. Concurrent processing: As with other sources, multiple dataset syncs run in parallel, each with its own isolated database session.

Authentication for Automated Sync

Dataset sync runs in the background with no user present, so 4MINDS replays the stored AWS connection to authenticate each cycle — exactly the same two-layer model used for interactive browsing:
  1. The stored auth method (IAM Role Federation or Cognito) is resolved into fresh temporary AWS credentials for the run (STS AssumeRoleWithWebIdentity, or Cognito GetCredentialsForIdentity).
  2. Those credentials mint a fresh temporary Redshift credential per table (GetClusterCredentials / GetCredentials).
No long-lived database password is ever stored. The IAM role (or Cognito authenticated role) must retain the Redshift IAM policy, and the resolved DB user must keep SELECT on the synced tables (see Database Access & Grants) — a revoked grant surfaces as a skipped table in the sync logs.

Networking Requirements

Redshift clusters and Serverless workgroups live inside a VPC. Only the 4MINDS backend opens SQL connections to Redshift — the browser app (app.4minds.ai) never connects to your database directly, so you only need to allow the backend’s egress IP. There is no frontend IP to allowlist. For 4MINDS to open a SQL connection, the endpoint must be reachable from the backend’s outbound network:
  • Allow the 4MINDS backend egress IP on the Redshift port (default 5439):
    EnvironmentBackend hostnameEgress IP to allowlist
    Productionapi.4minds.ai20.7.240.218/32
    Add it as an inbound rule on the security group attached to your cluster or Serverless workgroup. AWS CLI:
    aws ec2 authorize-security-group-ingress \
      --group-id <your-redshift-security-group-id> \
      --protocol tcp --port 5439 \
      --cidr 20.7.240.218/32
    
    Or in the console: EC2 → Security Groups → your Redshift SG → Inbound rules → Edit inbound rules → Add rule, then set Type = Custom TCP, Port range = 5439, Source = 20.7.240.218/32.
  • Publicly accessible = Yes is the simplest path for temporary testing; for production, prefer a VPC peering / PrivateLink setup with 4MINDS support.
  • SSL — All connections use SSL (sslmode=require), as required for temporary-credential auth.
Note: If a connection still times out after allowlisting the IP above, the backend egress IP may have changed — contact 4MINDS support to confirm the current address before widening your rules.
If the SQL connection times out, the browser shows:
“Connection timed out. The backend may not be able to reach the Redshift endpoint — check VPC / security group rules (inbound port open from the backend’s network).”

Security Model

CredentialWhere it livesHow long
AWS IAM Role ARN / Cognito config4MINDS database (encrypted metadata)Until you disconnect
Temporary AWS credentials (STS)Request memory only~1 hour, minted per call
Redshift DB credentials (Get(Cluster)Credentials)Request memory only~1 hour, generated per call
No database passwords are stored or transmitted. Each SQL operation mints a short-lived credential from the existing AWS credentials, opens a short-lived SQLAlchemy engine over SSL, runs one query, and disposes of the engine before returning.

Testing Your Connection

After saving credentials, the Test Connection button:
  1. Resolves your auth method → AWS credentials (via STS AssumeRoleWithWebIdentity for Role Federation, or Cognito GetCredentialsForIdentity for Cognito).
  2. Calls redshift:DescribeClusters with a 15-second timeout.
  3. Returns “Connection successful. Found N Redshift cluster(s).” on success.
Database-level authentication is tested when you browse a cluster or workgroup in the import wizard — it uses the same AWS credentials to mint a temporary credential and verify SQL connectivity.

Troubleshooting

IssueSolution
AccessDenied on Test ConnectionThe IAM policy isn’t attached to the role, or is missing redshift:DescribeClusters
Serverless workgroups don’t appearThe role is missing redshift-serverless:ListWorkgroups, or there are no workgroups in that region
Invalid Access Key ID / Invalid Secret Access KeyWrong static credentials (only relevant if using legacy access-keys auth)
Unrecognized clientWrong AWS region, or Redshift isn’t enabled in that region
Could not resolve Redshift endpoint hostnameThe endpoint in the list is wrong, or DNS can’t resolve it from the backend
Connection timed out (SQL step)Security group doesn’t allow inbound port 5439 from the 4MINDS backend egress IP (20.7.240.218/32) — see Networking Requirements
Connection refusedPort 5439 isn’t open on the security group, or the endpoint isn’t reachable and there’s no peering
Authentication failedThe IAM policy is missing redshift:GetClusterCredentials / redshift-serverless:GetCredentials, or the resolved DB user lacks a login/grant. See Database Access & Grants
permission denied for relation ...The DB user exists but lacks SELECT on the table — run the GRANT statements in Database Access & Grants
SSL error connecting to RedshiftTemporary-credential auth requires SSL — AWS-managed certificates work out of the box
Tables list is emptyThe database has no base tables, or the DB user lacks access to information_schema

Disconnecting

To remove the Redshift integration:
  1. Open IntegrationsAmazon Redshift → click Settings.
  2. Click Disconnect.
This deactivates the stored AWS connection metadata on 4MINDS. Your AWS resources (IAM roles, OIDC providers, Cognito pools, policies, and Redshift DB users) are untouched — delete them in the AWS Console if they’re no longer needed.