Skip to main content

Frequency Documentation

Configure Partner S3 Bucket for Studio Ingestion

In order to ingest via an S3 bucket, event configuration must be added to the bucket. For external buckets, Frequency must be able to add event notifications, receive events, and download objects for further processing.

  • Restrictions: The bucket must not have any existing event notifications configured. If there is an existing configuration, we will not be able to add our event notification, and the process will fail.

How to Check for Existing Event Notifications:

  1. Go to the AWS Console.

    Screenshot_2023-11-16_at_6_25_14_PM.png
  2. Select the S3 service.

    Screenshot_2023-11-16_at_6_25_39_PM.png
  3. Navigate to the bucket you want to configure.

    Screenshot_2023-11-16_at_6_26_14_PM.png
  4. Open the Properties section and check for Event Notifications. There should be no existing event notifications.

    Screenshot_2023-11-16_at_6_27_06_PM.png

Prerequisites for Studio Ingestion

  1. Check AWS Region: Verify the AWS region of the bucket.

  2. Verify SNS Topic:

    • Confirm with Frequency that an SNS topic named prd-freq-s3-ingestion-<region> exists in the region.

    • The SNS topic must have a subscription to the SQS queue with the ARN: arn:aws:sqs:us-west-2:077497804067:prd-freq-s3-inge-s3-events.

    • If the topic doesn’t exist, request that Frequency creates an SNS topic named prd-freq-s3-ingestion-<region> and subscribe it to the SQS ARN provided above.

How to Add a Policy to the Bucket:

  1. Go to the AWS Console.

    Screenshot_2023-11-16_at_6_25_14_PM.png
  2. Select the S3 service.

    Screenshot_2023-11-16_at_6_25_39_PM.png
  3. Navigate to the bucket you want to configure.

    Screenshot_2023-11-16_at_6_26_14_PM.png
  4. Open the Permissions tab and scroll to the Bucket Policy section.

    Screenshot_2023-11-16_at_6_50_47_PM.png
  5. Click Edit, paste the policy below, and save the changes.

    Screenshot_2023-11-16_at_6_51_32_PM.png

Policy to Add to the Bucket for Provisioning:

{
    "Version": "2012-10-17",
    "Id": "Policy1692967213597",
    "Statement": [
        {
            "Sid": "Frequency-S3-Ingestion-Allow-Access",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::077497804067:root"
            },
            "Action": [
                "s3:GetBucketNotification"
,               "s3:GetObject",
                "s3:ListBucket",
                "s3:PutBucketNotification",
                "s3:PutObject",
                "s3:GetBucketLocation"
            ],
            "Resource": [
                "<bucket_arn>",
                "<bucket_arn>/*"
            ]
        }
    ]
}

The bucket is now ready for Studio Ingestion.