S3 Binary Cache Store
Store URL format: s3://
bucket-name
This store allows reading and writing a binary cache stored in an AWS S3 (or S3-compatible service) bucket. This store shares many idioms with the HTTP Binary Cache Store.
For AWS S3, the binary cache URL for a bucket named example-nix-cache
will be exactly s3://example-nix-cache.
For S3 compatible binary caches, consult that cache's documentation.
Anonymous reads to your S3-compatible binary cache
If your binary cache is publicly accessible and does not require authentication, it is simplest to use the [HTTP Binary Cache Store] rather than S3 Binary Cache Store with https://example-nix-cache.s3.amazonaws.com instead of s3://example-nix-cache.
Your bucket will need a bucket policy like the following to be accessible:
{
"Id": "DirectReads",
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowDirectReads",
"Action": [
"s3:GetObject",
"s3:GetBucketLocation"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::example-nix-cache",
"arn:aws:s3:::example-nix-cache/*"
],
"Principal": "*"
}
]
}
Authentication
Nix will use the default credential provider chain for authenticating requests to Amazon S3.
Note that this means Nix will read environment variables and files with different idioms than with Nix's own settings, as implemented by the AWS SDK. Consult the documentation linked above for further details.
Authenticated reads to your S3 binary cache
Your bucket will need a bucket policy allowing the desired users to perform the s3:GetObject
and s3:GetBucketLocation
action on all objects in the bucket.
The anonymous policy given above can be updated to have a restricted Principal
to support this.
Authenticated writes to your S3-compatible binary cache
Your account will need an IAM policy to support uploading to the bucket:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "UploadToCache",
"Effect": "Allow",
"Action": [
"s3:AbortMultipartUpload",
"s3:GetBucketLocation",
"s3:GetObject",
"s3:ListBucket",
"s3:ListBucketMultipartUploads",
"s3:ListMultipartUploadParts",
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::example-nix-cache",
"arn:aws:s3:::example-nix-cache/*"
]
}
]
}
Examples
With bucket policies and authentication set up as described above, uploading works via nix copy
(experimental).
-
To upload with a specific credential profile for Amazon S3:
$ nix copy nixpkgs.hello \ --to 's3://example-nix-cache?profile=cache-upload®ion=eu-west-2'
-
To upload to an S3-compatible binary cache:
$ nix copy nixpkgs.hello --to \ 's3://example-nix-cache?profile=cache-upload&scheme=https&endpoint=minio.example.com'
Settings
-
Size (in bytes) of each part in multi-part uploads.
Default:
5242880
-
NAR compression method (
xz
,bzip2
,gzip
,zstd
, ornone
).Default:
xz
-
The preset level to be used when compressing NARs. The meaning and accepted values depend on the compression method selected.
-1
specifies that the default compression level should be used.Default:
-1
-
The URL of the endpoint of an S3-compatible service such as MinIO. Do not specify this setting if you're using Amazon S3.
Note
This endpoint must support HTTPS and will use path-based addressing instead of virtual host based addressing.
Default: empty
-
Whether to index DWARF debug info files by build ID. This allows
dwarffs
to fetch debug info on demandDefault:
false
-
Path to a local cache of NARs fetched from this binary cache, used by commands such as
nix store cat
.Default: empty
-
Compression method for
log/*
files. It is recommended to use a compression method supported by most web browsers (e.g.brotli
).Default: empty
-
Compression method for
.ls
files.Default: empty
-
Whether to use multi-part uploads.
Default:
false
-
Compression method for
.narinfo
files.Default: empty
-
Enable multi-threaded compression of NARs. This is currently only available for
xz
andzstd
.Default:
false
-
Size of the in-memory store path metadata cache.
Default:
65536
-
Priority of this store when used as a substituter. A lower value means a higher priority.
Default:
0
-
The name of the AWS configuration profile to use. By default Nix will use the
default
profile.Default: empty
-
The region of the S3 bucket. If your bucket is not in
us–east-1
, you should always explicitly specify the region parameter.Default:
us-east-1
-
The scheme used for S3 requests,
https
(default) orhttp
. This option allows you to disable HTTPS for binary caches which don't support it.Note
HTTPS should be used if the cache might contain sensitive information.
Default: empty
-
Path to the secret key used to sign the binary cache.
Default: empty
-
Logical location of the Nix store, usually
/nix/store
. Note that you can only copy store paths between stores if they have the samestore
setting.Default:
/nix/store
-
Optional system features available on the system this store uses to build derivations.
Example:
"kvm"
Default: machine-specific
-
Whether paths from this store can be used as substitutes even if they are not signed by a key listed in the
trusted-public-keys
setting.Default:
false
-
Whether this store can be queried efficiently for path validity when used as a substituter.
Default:
false
-
Whether to write a JSON file that lists the files in each NAR.
Default:
false