Tutorial: Giving a Single User Access to a Single Scaleway Object Storage (S3) Bucket

Text

Overview

This tutorial is a basic introduction to granting a single user access to a single S3 Bucket on the Scaleway Object Storage platform.


Quick summary for the impatient

If you're less of a beginner, this summary is for you. 
If all the technical jargon scares you, skip down to the introduction for the beginner-friendly full version.

In accordance with version 2023-04-17 of the Scaleway bucket policy:

  • For EVERY principal (user / app / group) that needs Object Storage, there MUST exist an IAM Policy that grants them the ObjectStorageFullAccess permission. This gives them full access to all buckets in the Project(s) indicated in the IAM Policy's scope.
  • You can optionally further restrict access on a bucket-by-bucket level: create a bucket policy for each bucket, in which you indicate the principal(s), action(s) and possibly condition(s) that apply for your use case.
  • Warning: AS SOON as a bucket policy exists for a given bucket, ONLY the principal(s) mentioned in that bucket policy retain access to the bucket. All other principals that may have had implicit access through the broad IAM policy before are now excluded  from accessing that bucket, until they're explixitly added to the bucket policy.
  • The owner of the Organization that owns a bucket can never be locked out of bucket access because they will ALWAYS retain the permissions to read and write bucket policies in their Organization.
     

Text

Introduction

Scaleway is a European internet infrastructure provider comparable with Amazon Web Services (AWS).

As a new user I struggled a bit to get a seemingly straightforward use case going, so here''s my write-up on how to give a single user or application access to a single bucket.

By extension it's actually a broader, but still very basic introduction to Scaleway IAM and Scaleway Object Storage.

Note 1:
Scaleway's Identity and Access Management (IAM for short) is modeled on AWS IAM, so the information may or may not apply to AWS S3 buckets as well. I don't know, I have no relevant experience with AWS.

Note 2:
The information in this article applies to the current Bucket Policy Version, which at the time of writing is 2023-04-17. You may find conflicting information elsewhere that was written for the now-deprecated Bucket Policy Version 2012-10-17. 

Text

Terminology

  • A policy is a set of rules that govern access to one or more resources, or permissions to perform certain actions.
  • A principal refers to which actor a policy applies to. A principal can refer to a user, an application, or a group.
  • A bucket is an Object Storage container, basically a kind of folder or directory that will contain your files. 
    (Note: Docker containers are something entirely different, and are unrelated to the storage we're talking about here)

Ok, with that out of the way, let's get to it.

Text

Steps

The steps we're going to go through:

  • Create an IAM principal.
  • Create an IAM policy to allow that principal access to Object Storage.
  • Create a bucket.
  • Create a bucket policy to lock that bucket down so only the newly created principal can access it.
Text

Create IAM principal

You must first create a principal (a user / application / group) that you want to grant Object Storage access to.

  • Go to Security & Identity > IAM
  • Create a new user, application, or group. 
  • Give it a useful name (e.g. "Bob", "Backup script", or "Editors")
  • Click Create. No need to worry about the policy or anything else just yet. Stick to the basics.

(In my case I wanted to set up an application principal to give a specific backup script of mine read/write permissions to store backups in a specific storage bucket.)

Text

Create IAM Policy

  • Go to Security & Identity > IAM > Policies and create a new policy.
  • Give the policy a useful name (e.g. "Storage Full Access") (don't worry, we will restrict that access later on).
  • On Assign Policy to Principal, assign it to the principal of your choice.
    (In my case I assigned it to my backup application)
  • Add the rule that grants full Object Storage access: 
    • search for "object", or drill down via Resource management > Object Storage, and choose ObjectStorageFullAccess.
  • Click Next.
  • Specify the project that will contain the bucket you're about to create.
  • Click Confirm rule.
  • Click Create Policy.

OK, at this point you have a principal and an IAM policy that allows that principal full Object Storage access to all current and future buckets under the specified Project, UNLESS there are also bucket policies in place that tighten access control.

That's right, the idea is that you first need to grant your principal broad access to Object Storage via an IAM policy, and then apply a bucket policy to each bucket to tighten access control.

Text

Create Bucket

  • Create a bucket via Storage > Object Storage > Create a bucket.
  • Choose a region.
  • Give the bucket a useful name (e.g. "Research backups")
  • Choose Private or Public, depending on your needs
  • Leave all the rest as-is for now (can be changed later).
  • Click Create bucket.

You can now upload files and folders via the browser.

In my case I want a backup script on my laptop to use the bucket, rather than uploading files manually I just created. My script will use the credentials of the "Backup script" application principal I created earlier.

Text

Tighten bucket access by setting a bucket policy

My requirements:

  • I want my backup script to ONLY have access to the backup bucket, and not to any other buckets in my Project. If somehow the credentials would get compromised (stolen, published, ...), I want the potential damage to be limited to only the contents of the backup bucket. NO other buckets should be accessible with those credentials.
  • I ONLY want my backup script to be able to access the backup bucket. NO other principals with Object Storage access should have access to the backup bucket.

Here's where things got confusing for me the first time around. You may want to re-read the following explanation a few times :)

A this point my "Backup script" principal already implicitly has full Object Storage access to all the buckets under my Project, because of the IAM policy I created at the beginning.

The next thing I now need to do is create a bucket policy that explicitly grants my principal the required permissions to the backup bucket.

I know that seems like overkill, but stay with me.

  • Go to Storage > Object Storage > Create bucket policy.
  • Provide a useful name (e.g. "Backup script policy").
  • Select your bucket.
  • Add a statement using the Policy generator (default option).
  • Leave Maintain access to bucket checked (!)
  • Provide a statement to give your principal full access:
    • Give the statement a useful name (e.g. "Grant access to backup script").
    • Select which type(s) of principal to include (e.g. "application").
    • Select which principal(s) to include (e.g. "Backup script").
    • Select All Actions (you can this refine later if necessary).
  • Click Create bucket policy.
    (if the button is not clickable, you forgot to fill in a required field)

That's it. Your bucket policy is in place for the specific bucket you selected.

Why did I need to set a bucket policy in additon to an IAM policy?

Setting a bucket policy on a specific bucket targeting one or more specific principal(s) has the following effect:

  • Because there now exists an explicit bucket policy, ONLY the mentioned principal(s) get access to the backup bucket. ALL other principals that may have had implicit access through the broad IAM policy before are now excluded. In other words, at this point ONLY the principal(s) mentioned in the bucket policy retain access. Part of my requirements is now satisfied.

The other part is that I want my principal to ONLY have access to the backup bucket and NOT to any other buckets in my Project.

To do this I have to also create a bucket policy for ALL of the other buckets in my Project and specify which principal(s) should have access.

Here's the reasoning:

Asuming people want to start out with basic requirements, granting all principals full Object Storage access (if they need it, of course) through an IAM policy means you don't need to start configuring things on bucket-by-bucket basis. Unless, of course, this is what you want, in which case, you can do so. But not everyone needs this, so Scaleway tries not to force everyone through bucket policies if it can be avoided.
 

If you're skeptical, try setting bucket policies without setting an IAM policy. Your principal won't have access to the overall Object Storage "space" to begin with, no matter which fine-grained permissions you grant on bucket level.

Remember, you don't have to grant full permissions on bucket level. You should learn about the available bucket-level permissions and strictly grant the smallest possible set of permissions for your principal to be able to do its job.

This is called The Principle of Least Access (no pun intended).

Final note:
Regardless of any bucket policies you may set up, the owner of the Organization will ALWAYS retain the right to create and delete bucket policies. In other words, the owner of an organization can NEVER be permanently locked of any bucket in the organisation because they will ALWAYS be ably to modify or delete all bucket policies.

Text

Next optional step: API keys

The missing piece of my requirements was generating API keys so my backup script can use the Scaleway API to access the bucket to store files. 

I'm not going to cover all of that here except to say:

  • A principal's API keys are managed at Security and Identity > IAM > API keys.
  • I use the opensource backup tool duplicity for all my backups. It needs a set of Scaleway API keys, and can optionally encrypt the contents so even Scaleway itself can ever access the content of the files in my buckets.
Text

Summary

In accordance with version 2023-04-17 of the Scaleway bucket policy:

  • For EVERY principal (user / app / group) that needs Object Storage, there MUST exist an IAM Policy that grants them the ObjectStorageFullAccess permission. This gives them full access to all buckets in the Project(s) indicated in the IAM Policy's scope.
  • You can optionally further restrict access on a bucket-by-bucket level: create a bucket policy for each bucket, in which you indicate the principal(s), action(s) and possibly condition(s) that apply for your use case.
  • Warning: AS SOON as a bucket policy exists for a given bucket, ONLY the principal(s) mentioned in that bucket policy retain access to the bucket. All other principals that may have had implicit access through the broad IAM policy before are now excluded  from accessing that bucket, until they're explixitly added to the bucket policy.
  • The owner of the Organization that owns a bucket can never be locked out of bucket access because they will ALWAYS retain the permissions to read and write bucket policies in their Organization.
Text

Further Reading

Scaleway Object Storage overview:
https://www.scaleway.com/en/docs/object-storage/ 

Scaleway IAM Permission sets:
https://www.scaleway.com/en/docs/iam/reference-content/permission-sets/#storage 

Scaleway bucket policy actions and resources:
https://www.scaleway.com/en/docs/object-storage/api-cli/bucket-policy 

Combining IAM and bucket policies to set up granular access to Object Storage:
https://www.scaleway.com/en/docs/object-storage/api-cli/combining-iam-and-object-storage/ 

Scaleway API Documentation:
https://www.scaleway.com/en/developers/api