> ## Documentation Index > Fetch the complete documentation index at: https://docs.4minds.ai/llms.txt > Use this file to discover all available pages before exploring further. # Amazon Redshift > Connect 4MINDS to Amazon Redshift — provisioned clusters and Redshift Serverless — to browse databases, preview tables, and import data as datasets, using IAM Role Federation or Amazon Cognito for AWS auth and temporary Redshift credentials for SQL access. This guide walks you through connecting 4MINDS to Amazon Redshift. 4MINDS supports both **provisioned clusters** and **Redshift Serverless workgroups** — they appear together in the same picker. The connection has **two layers**: 1. **AWS-level authentication** — how 4MINDS discovers which Redshift clusters and Serverless workgroups exist in your account. Uses **IAM Role Federation** (recommended) or **Amazon Cognito**, both shared with the rest of 4MINDS's AWS integrations. 2. **Database-level authentication** — uses **temporary Redshift credentials** minted from the same AWS credentials used for discovery. Provisioned clusters use `redshift:GetClusterCredentials`; Serverless workgroups use `redshift-serverless:GetCredentials`. No long-lived database password is stored. This two-layer model lets 4MINDS list your Redshift resources and connect to individual databases with a single set of AWS credentials — no long-lived database passwords to manage or rotate. For the generic IAM Role / Cognito setup (creating the OIDC provider, trust policy, Cognito user pool, etc.), see [AWS Integrations](/aws-integrations). The steps below focus on what's **specific to Redshift**. *** ## Provisioned vs. Serverless | | Provisioned cluster | Redshift Serverless | | ------------------ | ------------------------------------------------------------ | ------------------------------------------------------------------------- | | Discovery API | `redshift:DescribeClusters` | `redshift-serverless:ListWorkgroups` | | Credential API | `redshift:GetClusterCredentials` | `redshift-serverless:GetCredentials` | | Database user | You choose the DB user (see [Database User](#database-user)) | Derived automatically from the IAM identity — **no DB user to configure** | | Default port | `5439` | `5439` | | Shown in picker as | Node type + node count | `Serverless · ` | Both types are listed together when you browse Redshift in the import wizard. The difference is handled server-side; you don't have to pick a mode. *** ## Redshift IAM Permissions Policy Whichever AWS auth method you use (IAM Role Federation or Cognito), attach this policy to the role. It covers **both** provisioned and Serverless. If you only use one, you can drop the statements for the other. ```json theme={null} { "Version": "2012-10-17", "Statement": [ { "Sid": "RedshiftProvisionedDiscovery", "Effect": "Allow", "Action": [ "redshift:DescribeClusters" ], "Resource": "*" }, { "Sid": "RedshiftProvisionedCredentials", "Effect": "Allow", "Action": "redshift:GetClusterCredentials", "Resource": [ "arn:aws:redshift:*:*:dbuser:*/*", "arn:aws:redshift:*:*:dbname:*/*" ] }, { "Sid": "RedshiftServerlessDiscovery", "Effect": "Allow", "Action": [ "redshift-serverless:ListWorkgroups" ], "Resource": "*" }, { "Sid": "RedshiftServerlessCredentials", "Effect": "Allow", "Action": "redshift-serverless:GetCredentials", "Resource": "*" } ] } ``` | Permission | Purpose | | ------------------------------------ | --------------------------------------------------------------------------------------------------------------------- | | `redshift:DescribeClusters` | Lists provisioned clusters in the connected region so 4MINDS can populate the picker (also backs **Test Connection**) | | `redshift:GetClusterCredentials` | Mints short-lived DB credentials for a provisioned cluster + DB user | | `redshift-serverless:ListWorkgroups` | Lists Serverless workgroups in the connected region | | `redshift-serverless:GetCredentials` | Mints short-lived DB credentials for a Serverless workgroup | Name the policy something memorable like `4MINDS-Redshift-Access` — you'll attach it when creating the IAM role (for Role Federation) or the Cognito authenticated role. > **Note:** `GetClusterCredentials` is authorized against both the `dbuser:` and `dbname:` resource ARNs. The wildcards above allow all clusters, DB users, and databases. To restrict, narrow the ARNs (e.g., `arn:aws:redshift:us-east-1:123456789012:dbuser:my-cluster/fourminds_iam_user`). ### Least-Privilege: Restricting to Specific Resources To expose only a subset of your Redshift resources, scope the discovery and credential ARNs: ```json theme={null} { "Effect": "Allow", "Action": "redshift:GetClusterCredentials", "Resource": [ "arn:aws:redshift:us-east-1:123456789012:dbuser:prod-analytics/fourminds_iam_user", "arn:aws:redshift:us-east-1:123456789012:dbname:prod-analytics/dev" ] }, { "Effect": "Allow", "Action": "redshift-serverless:GetCredentials", "Resource": "arn:aws:redshift-serverless:us-east-1:123456789012:workgroup/*" } ``` *** ## Database Access & Grants The IAM identity 4MINDS assumes must map to a database user that has read access to the tables you want to import. ### Provisioned clusters — the DB user For a provisioned cluster, 4MINDS calls `GetClusterCredentials` with a **DB username** you configure (see [Database User](#database-user)). That user must already exist in the cluster and have `SELECT` on the target tables. `AutoCreate` is **not** used — 4MINDS never creates users in your cluster. ```sql theme={null} -- run in the target database on your Redshift cluster CREATE USER fourminds_iam_user PASSWORD DISABLE; GRANT USAGE ON SCHEMA public TO fourminds_iam_user; GRANT SELECT ON ALL TABLES IN SCHEMA public TO fourminds_iam_user; -- so future tables are readable too ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO fourminds_iam_user; ``` The IAM policy's `dbuser:` ARN must match this username (e.g., `.../my-cluster/fourminds_iam_user`). ### Serverless workgroups — IAM-derived user Redshift Serverless `GetCredentials` does **not** accept a DB username — it derives the database user from the IAM identity making the call (returned as `IAMR:` or `IAM:`). To grant it access, create/grant that exact user in the database: ```sql theme={null} -- is the IAM role 4MINDS assumes (Role Federation) or the -- Cognito authenticated role. Redshift prefixes assumed roles with "IAMR:". CREATE USER "IAMR:4MINDS-integration-role" PASSWORD DISABLE; GRANT USAGE ON SCHEMA public TO "IAMR:4MINDS-integration-role"; GRANT SELECT ON ALL TABLES IN SCHEMA public TO "IAMR:4MINDS-integration-role"; ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO "IAMR:4MINDS-integration-role"; ``` > **Tip:** If you're unsure of the exact derived username, run a browse in 4MINDS and check the error message — a `permission denied` failure will name the DB user Redshift resolved, which you can then `GRANT` to. > **Note:** For Serverless, `GetCredentials` **auto-creates** the `IAMR:` database user on first use, so the explicit `CREATE USER` above is optional — granting `SELECT` to `PUBLIC` (or to the resolved user after a first browse) is sufficient. Pre-creating the user is only needed if you want to grant privileges before the first connection. *** ## Connection Method 1: IAM Role Federation (Recommended) Short-lived credentials minted per request through AWS STS — no long-lived AWS keys stored in 4MINDS. 1. Complete the one-time AWS setup in [AWS Integrations → IAM Role Federation](/aws-integrations#connection-method-1-iam-role-federation-recommended) (OIDC provider + IAM role with trust policy). 2. Attach the **Redshift IAM policy** above to the role (Role Federation and Cognito can share the same role — just attach both service policies to it if you use multiple AWS integrations). 3. In 4MINDS, open **Integrations** → **Amazon Redshift** → click **Configure** (or **Settings** if already connected). 4. Keep the **IAM Role** tab selected. 5. Paste your **IAM Role ARN** (e.g., `arn:aws:iam::123456789012:role/4MINDS-integration-role`). 6. Leave the **External ID** field blank — it is not supported for IAM Role Federation (see [AWS Integrations → Verify the Trust Policy](/aws-integrations#c-verify-the-trust-policy)). 7. Enter your **AWS Region** (e.g., `us-east-1`) — this must match the region of the Redshift resources you want to browse. 8. *(Provisioned only)* Enter a **Database user** — see [Database User](#database-user). Leave blank to use the default `fourminds_iam_user`. 9. Click **Test Connection** — success shows *"Connection successful. Found N Redshift cluster(s)."* 10. Click **Save Credentials**. *** ## Connection Method 2: Amazon Cognito Use this if your organization already authenticates against AWS through Cognito User Pools and Identity Pools. 1. Complete the one-time Cognito setup in [AWS Integrations → Amazon Cognito](/aws-integrations#connection-method-2-amazon-cognito) (User Pool, App Client, Identity Pool, authenticated role). 2. Attach the **Redshift IAM policy** above to the Cognito **authenticated role**. 3. In 4MINDS, open **Integrations** → **Amazon Redshift** → click **Configure**. 4. Switch to the **Cognito** tab. 5. Fill in **User Pool ID**, **App Client ID**, **App Client Secret** (if configured), **Identity Pool ID**, **Username**, and **Password**. 6. Enter your **AWS Region**. 7. *(Provisioned only)* Enter a **Database user** if you're not using the default. 8. Click **Test Connection**, then **Save Credentials**. *** ## Database User `GetClusterCredentials` (provisioned clusters only) authenticates as a specific database user. In the settings modal, the **Database user** field controls which user 4MINDS requests credentials for: * Defaults to `fourminds_iam_user` if left blank. * The user **must already exist** in the cluster with `SELECT` on the tables you want to import (see [Database Access & Grants](#database-access--grants)). * The value is stored once with your connection metadata and reused for every browse and import — you don't re-enter it per request. * **Ignored for Serverless** — Serverless derives the user from the IAM identity, so this field has no effect on workgroups. *** ## 4MINDS Fields | Field | Required | Notes | | ------------------ | --------------- | ------------------------------------------------------------------------------------------------------------------- | | **AWS Region** | Yes | Region where your Redshift resources are deployed. Must match the region of your Cognito resources (Cognito method) | | **IAM Role ARN** | IAM Role method | Role with the Redshift IAM policy attached | | **External ID** | No | Not supported for IAM Role Federation — leave blank | | **Database user** | No | Provisioned clusters only. Defaults to `fourminds_iam_user`. Ignored for Serverless | | **Cognito fields** | Cognito method | See [AWS Integrations](/aws-integrations#gather-your-cognito-details) | *** ## Importing Data from Redshift Once connected, open the **Datasets** page and click **Add Source** → **Amazon Redshift** (or pick Redshift from the data source bar inside an existing dataset). The import wizard walks you through the hierarchy: **clusters/workgroups → databases → tables**. ### Step 1 — Pick a Cluster or Workgroup 4MINDS calls `redshift:DescribeClusters` **and** `redshift-serverless:ListWorkgroups` in your configured region and shows everything the role can see. Provisioned clusters show their node type and node count; Serverless workgroups are labeled `Serverless` with their namespace. Click one to continue. > If the Serverless permissions are missing, workgroups are simply omitted — provisioned clusters still list normally. ### Step 2 — Browse Databases 4MINDS mints a short-lived credential (via `GetClusterCredentials` for a cluster, or `GetCredentials` for a workgroup), opens a live SQL connection to the endpoint over SSL, and lists user databases. System databases (`template0`, `template1`, `padb_harvest`, `rdsadmin`) are filtered out. ### Step 3 — Select Tables Click a database to list its base tables. For each table the browser shows: * Schema-qualified name (e.g., `public.customers`) * Column count * On-disk size (when available) Check the tables you want to import and click **Add Tables**. Selected tables are staged into your dataset alongside files from any other source. ### During Import At import time, 4MINDS runs a bulk fetch on each selected table: * Up to **50,000 rows** per table (hard cap — larger tables are truncated and flagged). * **Binary columns** (`BYTEA`, `VARBINARY`, `BINARY`) are dropped from the export rather than base64-encoded. * Complex types (arrays, structured values) are serialized with `json.dumps` into a single cell. * Timeout is **120 seconds** per table. Each table lands in your dataset as a UTF-8 CSV with the original column names as headers. *** ## Dataset Sync ### Overview Beyond one-time imports, 4MINDS can keep a Redshift-backed dataset up to date automatically. When you enable **Dataset Sync** on a dataset built from Redshift tables, 4MINDS re-exports the configured tables on a schedule — no manual re-imports. Unlike file-based sources (S3, Dropbox, Databricks volumes), Redshift is a **table warehouse** with no cheap per-table "last modified" signal. So Redshift sync follows the same **full-snapshot** model as BigQuery rather than incremental file diffing: **each sync cycle re-exports the whole table** (up to the 50,000-row cap) and replaces the previous snapshot. There is no row-level change detection — every run is a fresh point-in-time copy. ### How to Set Up Sync 1. Import one or more Redshift tables into a 4MINDS dataset (using the import wizard above). 2. On the dataset, toggle **Dataset Sync** on. 3. Select a sync frequency (see table below). 4. From that point on, 4MINDS re-exports each configured table at the chosen interval. A single sync configuration can span **multiple clusters, workgroups, and databases** — each table carries its own connection target (endpoint, port, serverless flag, schema), so tables from different Redshift sources can live in the same synced dataset. ### Sync Frequencies | Frequency | Interval | Best For | | ---------------- | -------- | -------------------------------------------------------- | | **Every minute** | 1 minute | Rapidly changing tables (Enterprise tier) | | **Hourly** | 1 hour | Frequently updated warehouse tables (Teams & Enterprise) | | **Daily** | 24 hours | Standard business reporting (All paid tiers) | | **Weekly** | 7 days | Slowly changing reference data | | **Monthly** | 30 days | Compliance snapshots, archival data | > Because every Redshift sync is a full-table re-export, prefer **less frequent** intervals for large tables to control export cost and RPU/credit usage. Daily or weekly is a good default for warehouse tables. ### How It Works Internally **Full snapshot, every cycle:** Because Redshift exposes no per-table modification timestamp, the sync manifest records no `modified_at` for Redshift entries. As a result, the writer treats every configured table as changed on every run and re-downloads a fresh snapshot — the same semantic BigQuery uses. **Stable table identity:** Each table is tracked by a stable identifier of the form `rs----` (schema defaults to `public`). The manifest de-duplicates on this id, so re-running a sync replaces the prior snapshot of a table rather than accumulating duplicates. **Export pipeline:** On each cycle, for every configured table 4MINDS mints a short-lived Redshift credential (via `GetClusterCredentials` for a cluster, or `GetCredentials` for a workgroup), opens an SSL SQL connection, and exports the table to CSV bytes — up to **50,000 rows** (the same cap as one-shot import; larger tables are truncated and flagged). Binary columns are dropped and empty tables are skipped. The CSV is uploaded to the user's dataset storage and processed through the 4MINDS ETL pipeline. **Concurrent processing:** As with other sources, multiple dataset syncs run in parallel, each with its own isolated database session. ### Authentication for Automated Sync Dataset sync runs in the background with no user present, so 4MINDS replays the **stored AWS connection** to authenticate each cycle — exactly the same two-layer model used for interactive browsing: 1. The stored auth method (IAM Role Federation or Cognito) is resolved into **fresh temporary AWS credentials** for the run (STS `AssumeRoleWithWebIdentity`, or Cognito `GetCredentialsForIdentity`). 2. Those credentials mint a **fresh temporary Redshift credential** per table (`GetClusterCredentials` / `GetCredentials`). No long-lived database password is ever stored. The IAM role (or Cognito authenticated role) must retain the [Redshift IAM policy](#redshift-iam-permissions-policy), and the resolved DB user must keep `SELECT` on the synced tables (see [Database Access & Grants](#database-access--grants)) — a revoked grant surfaces as a skipped table in the sync logs. *** ## Networking Requirements Redshift clusters and Serverless workgroups live inside a VPC. Only the **4MINDS backend** opens SQL connections to Redshift — the browser app (`app.4minds.ai`) never connects to your database directly, so you only need to allow the **backend's egress IP**. There is no frontend IP to allowlist. For 4MINDS to open a SQL connection, the endpoint must be reachable from the backend's outbound network: * **Allow the 4MINDS backend egress IP** on the Redshift port (default `5439`): | Environment | Backend hostname | Egress IP to allowlist | | ----------- | ---------------- | ---------------------- | | Production | `api.4minds.ai` | `20.7.240.218/32` | Add it as an inbound rule on the security group attached to your cluster or Serverless workgroup. AWS CLI: ```bash theme={null} aws ec2 authorize-security-group-ingress \ --group-id \ --protocol tcp --port 5439 \ --cidr 20.7.240.218/32 ``` Or in the console: **EC2 → Security Groups →** your Redshift SG **→ Inbound rules → Edit inbound rules → Add rule**, then set **Type** = `Custom TCP`, **Port range** = `5439`, **Source** = `20.7.240.218/32`. * **Publicly accessible = Yes** is the simplest path for temporary testing; for production, prefer a VPC peering / PrivateLink setup with 4MINDS support. * **SSL** — All connections use SSL (`sslmode=require`), as required for temporary-credential auth. > **Note:** If a connection still times out after allowlisting the IP above, the backend egress IP may have changed — contact 4MINDS support to confirm the current address before widening your rules. If the SQL connection times out, the browser shows: > *"Connection timed out. The backend may not be able to reach the Redshift endpoint — check VPC / security group rules (inbound port open from the backend's network)."* *** ## Security Model | Credential | Where it lives | How long | | --------------------------------------------------- | ------------------------------------ | ---------------------------- | | AWS IAM Role ARN / Cognito config | 4MINDS database (encrypted metadata) | Until you disconnect | | Temporary AWS credentials (STS) | Request memory only | \~1 hour, minted per call | | Redshift DB credentials (`Get(Cluster)Credentials`) | Request memory only | \~1 hour, generated per call | No database passwords are stored or transmitted. Each SQL operation mints a short-lived credential from the existing AWS credentials, opens a short-lived SQLAlchemy engine over SSL, runs one query, and disposes of the engine before returning. *** ## Testing Your Connection After saving credentials, the **Test Connection** button: 1. Resolves your auth method → AWS credentials (via STS AssumeRoleWithWebIdentity for Role Federation, or Cognito GetCredentialsForIdentity for Cognito). 2. Calls `redshift:DescribeClusters` with a 15-second timeout. 3. Returns *"Connection successful. Found N Redshift cluster(s)."* on success. Database-level authentication is tested when you browse a cluster or workgroup in the import wizard — it uses the same AWS credentials to mint a temporary credential and verify SQL connectivity. *** ## Troubleshooting | Issue | Solution | | ----------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `AccessDenied` on Test Connection | The IAM policy isn't attached to the role, or is missing `redshift:DescribeClusters` | | Serverless workgroups don't appear | The role is missing `redshift-serverless:ListWorkgroups`, or there are no workgroups in that region | | `Invalid Access Key ID` / `Invalid Secret Access Key` | Wrong static credentials (only relevant if using legacy access-keys auth) | | `Unrecognized client` | Wrong AWS region, or Redshift isn't enabled in that region | | `Could not resolve Redshift endpoint hostname` | The endpoint in the list is wrong, or DNS can't resolve it from the backend | | `Connection timed out` (SQL step) | Security group doesn't allow inbound port `5439` from the 4MINDS backend egress IP (`20.7.240.218/32`) — see [Networking Requirements](#networking-requirements) | | `Connection refused` | Port `5439` isn't open on the security group, or the endpoint isn't reachable and there's no peering | | `Authentication failed` | The IAM policy is missing `redshift:GetClusterCredentials` / `redshift-serverless:GetCredentials`, or the resolved DB user lacks a login/grant. See [Database Access & Grants](#database-access--grants) | | `permission denied for relation ...` | The DB user exists but lacks `SELECT` on the table — run the `GRANT` statements in [Database Access & Grants](#database-access--grants) | | `SSL error connecting to Redshift` | Temporary-credential auth requires SSL — AWS-managed certificates work out of the box | | Tables list is empty | The database has no base tables, or the DB user lacks access to `information_schema` | *** ## Disconnecting To remove the Redshift integration: 1. Open **Integrations** → **Amazon Redshift** → click **Settings**. 2. Click **Disconnect**. This deactivates the stored AWS connection metadata on 4MINDS. Your AWS resources (IAM roles, OIDC providers, Cognito pools, policies, and Redshift DB users) are untouched — delete them in the AWS Console if they're no longer needed.