Skip to main content
DuckLake is a lakehouse format that keeps table metadata in a SQL catalog database and stores the underlying data as files in object storage. To connect DuckLake to Credible you provide both: a catalog connection for metadata and a storage connection for data.

Prerequisites

  • A catalog database for DuckLake metadata — currently PostgreSQL is supported
  • An object storage location for the data — Amazon S3 or Google Cloud Storage (GCS)
  • Credentials with read access to both the catalog database and the storage bucket

Step 1: Prepare the Catalog (PostgreSQL)

DuckLake stores its table metadata in a PostgreSQL database. You can point at an existing database or create a dedicated one. Have the following ready for the connection:
  • Host and Port (default 5432)
  • Database Name
  • Username and Password
You can supply these fields individually or as a single PostgreSQL connection string (postgresql://username:password@hostname:port/database).

Step 2: Prepare Storage (S3 or GCS)

DuckLake reads and writes its data files in object storage. Choose S3 or GCS and gather credentials. For Amazon S3:
  • Bucket URL — e.g. s3://my-bucket/path
  • Access Key ID and Secret Access Key
  • Optional: Region, a Custom Endpoint (for S3-compatible stores), and a Session Token for temporary STS credentials
Grant the credentials read access to the bucket and prefix that hold your DuckLake data. Write access is only needed if you plan to materialize data back to the lake.

Step 3: Connect in Credible

  1. Go to your-org.app.credibledata.com
  2. Navigate to your environment
  3. Click Add Connection
  4. Select DuckLake
  5. Under Catalog, choose PostgreSQL and enter the catalog database details from Step 1
  6. Under Storage, choose S3 or GCS and enter the storage details from Step 2
  7. Click Test Connection to verify both the catalog and storage are reachable
  8. Save the connection

Troubleshooting

  • Catalog connection fails — Confirm the PostgreSQL host, port, and credentials are correct and that the database is reachable from the Credible service.
  • Storage access denied — Check the bucket URL and that the access key can list and read objects under the given prefix. For S3-compatible stores, set the Custom Endpoint.
  • Temporary credentials expired — If you used a Session Token, STS credentials are short-lived; regenerate them or switch to a long-lived key.