rez.package_cache

class rez.package_cache.PackageCache(path)[source]

Bases: object

Package cache.

A package cache is responsible for storing copies of variant payloads into a location that would typically be on local disk. The intent is to avoid fetching a package’s files over shared storage at runtime.

A package cache is used like so:

  • A rez-env is performed;

  • The context is resolved;

  • For each variant in the context, we check to see if it’s present in the current package cache;

  • If it is, the variant’s root is remapped to this location.

A package cache is _not_ a package repository. It just stores copies of variant payloads - no package definitions are stored.

Payloads are stored into the following structure:

/<cache_dir>/foo/1.0.0/af8d/a/<payload>

/a.json

Here, ‘af8d’ is the first 4 chars of the SHA1 hash of the variant’s ‘handle’, which is a dict of fields that uniquely identify the variant. To avoid hash collisions, the variant is then stored under a subdir that is incrementally named (‘a’, ‘b’, …, ‘aa’, ‘ab’, …). The ‘a.json’ file is used to find the correct variant within the hash subdir. The intent is to keep cached paths short, and avoid having to search too many variant.json files to find the matching variant.

VARIANT_COPYING = 3
VARIANT_COPY_STALLED = 4
VARIANT_CREATED = 2
VARIANT_FOUND = 1
VARIANT_NOT_FOUND = 0
VARIANT_PENDING = 5
VARIANT_REMOVED = 6
add_variant(variant, force=False)[source]

Copy a variant’s payload into the cache.

The following steps are taken to ensure muti-thread/proc safety, and to guarantee that a partially-copied variant payload is never able to be used:

  1. The hash dir (eg ‘/<cache_dir>/foo/1.0.0/af8d’) is created;

  2. A file lock mutex (‘/<cache_dir>/.lock’) is acquired;

  3. The file ‘/<cache_dir>/foo/1.0.0/af8d/.copying-a’ (or -b, -c etc) is created. This tells rez that this variant is being copied and cannot be used yet;

  4. The file ‘/<cache_dir>/foo/1.0.0/af8d/a.json’ is created. Now another proc/thread can’t create the same local variant;

  5. The file lock is released;

  6. The variant payload is copied to ‘/<cache_dir>/foo/1.0.0/af8d/a’;

  7. The ‘.copying-a’ file is removed.

Note that the variant will not be cached in the following circumstances, unless force is True:

  • The variant is not cachable as determined by Variant.is_cachable;

  • The variant is from a local package, and ‘config.package_cache_local’ is False;

  • The variant is stored on the same disk device as this cache, and config.package_cache_same_device’ is False.

Parameters

  • variant (Variant) – The variant to copy into this cache

  • force (bool) – Copy the variant regardless. Use at your own risk (there is no guarantee the resulting variant payload will be functional).

Returns

  • str: Path to cached payload

  • int: One of: - VARIANT_FOUND - VARIANT_CREATED - VARIANT_COPYING - VARIANT_COPY_STALLED

Return type

2-tuple

add_variants_async(variants)[source]

Update the package cache by adding some or all of the given variants.

This method is called when a context is created or sourced. Variants are then added to the cache in a separate process.

clean(time_limit=None)[source]

Delete unused package cache files.

This should be run periodically via ‘rez-pkg-cache –clean’.

This removes: - Variants that have not been used in more than

‘config.package_cache_max_variant_days’ days;

  • Variants that have stalled;

  • Variants that are already pending deletion (remove_variant() was used).

Parameters

time_limit (float) – Perform cleaning operations only up until this limit, resulting in a possibly incomplete cleanup. This is used to keep the cache size down without having to periodically run ‘rez-pkg-cache –clean’.

get_cached_root(variant)[source]

Get location of variant payload copy.

Parameters

variant (Variant) – Variant to search for.

Returns

Cached variant root path, or None if not found.

Return type

str

get_variants()[source]

Get variants and their current statuses from the cache.

Returns

  • Variant: The cached variant

  • str: Local cache path for variant, if determined (’’ otherwise)

  • int: Status. One of: - VARIANT_FOUND - VARIANT_COPYING - VARIANT_COPY_STALLED - VARIANT_PENDING

Return type

List of 3-tuple

remove_variant(variant)[source]

Remove a variant from the cache.

Since this removes the associated cached variant payload, there is no guarantee that this will not break packages currently in use by a context.

Note that this does not actually free up associated disk space - you must call clean() to do that.

Returns

One of: - VARIANT_REMOVED - VARIANT_NOT_FOUND - VARIANT_COPYING

Return type

int

run_daemon()[source]

Run as daemon and copy pending variants.

Called via rez-pkg-cache –daemon.