Python Client¶
Constructors¶
These are functions for constructing a client object.
|
Connect to a Node on a local or remote server. |
|
Build a Node based a 'profile' (a named configuration). |
Client Node¶
The Node interface extends the collections.abc.Mapping
(i.e. read-only
dict) interface, so it supports these standard “magic methods”:
__getitem__
(lookup by key with[]
)__iter__
(iteration, use in for-loops for example)__len__
(has a length, can be passed tolen
)
as well as:
|
|
The views returned by .keys()
, .items()
, and .values()
support efficient random access—e.g.
node.values()[3]
node.values()[-1]
node.values()[:3]
and several convenience methods:
Get the first value. |
|
Get the last value. |
|
Get the first N values. |
|
Get the last N values. |
Likewise for .keys()
and .items()
.
Beyond the Mapping interface, Node adds the following attributes
Metadata about this data source. |
|
References (links) to related context, metadata, or data. |
|
The current sorting of this Node |
|
Direct link to this entry |
|
List of specifications describing the structure of the metadata and/or data. |
It adds these methods, which return a new Node instance.
Make a Node with a subset of this Node's entries, filtered by query. |
|
|
Make a Node with the same entries but sorted according to sorting. |
It adds these methods for downloading and refreshing cached data.
Access all the data in this Node. |
|
|
Refresh cached data for this node. |
It adds this method, which returns the unique metadata keys, structure_families, and specs of its children along with their counts.
|
Get the unique values and optionally counts of metadata_keys, structure_families, and specs in this Node's entries |
Structure Clients¶
For each structure family (“array”, “dataframe”, etc.) there is a client object that understand how to request and decode chunks/partitions of data for this structure.
In fact, there can be more than one client for a given structure family. Tiled currently includes two clients for each structure family:
A client that reads the data into dask-backed objects (dask array, dask DataFrame, xarray objects backed by dask arrays)
A client that reads the data into in-memory structures (numpy array, pandas DataFrame, xarray objects backed by numpy arrays)
Base¶
|
|
List formats that the server can export this data as. |
|
Metadata about this data source. |
|
Direct link to this entry |
|
JSON payload describing this item. |
|
Depending on the server's authentication method, this will prompt for username/password: |
|
Log out. |
|
This is intended primarily for internal use and use by subclasses. |
|
List of specifications describing the structure of the metadata and/or data. |
|
Download all data into the cache. |
|
Refresh cached data for this node. |
|
Return a dataclass describing the structure of the data. |
Array¶
|
Client-side wrapper around an array-like that returns dask arrays |
Access the data for one block of this chunked (dask) array. |
|
Acess the entire array or a slice. |
|
Download data in some format and write to a file. |
|
Client-side wrapper around an array-like that returns in-memory arrays |
Access the data for one block of this chunked array. |
|
|
Acess the entire array or a slice. |
Download data in some format and write to a file. |
Sparse Array¶
|
|
Download data in some format and write to a file. |
DataFrame¶
Client-side wrapper around an dataframe-like that returns dask dataframes |
|
|
Access one partition in a partitioned (dask) dataframe. |
Access the entire DataFrame. |
|
Download data in some format and write to a file. |
Client-side wrapper around a dataframe-like that returns in-memory dataframes |
|
Access one partition of the DataFrame. |
|
Access the entire DataFrame. |
|
Download data in some format and write to a file. |
Xarray Dataset¶
|
|
|
|
Cache¶
The module tiled.client.cache
includes objects inspired by https://github.com/dask/cachey/
We opted for an independent implementation because reusing cachey would have required:
An invasive subclass that could be a bit fragile
And also composition in order to get the public API we want
Carrying around some complexity/features that we do not use here
The original cachey license (which, like Tiled’s, is 3-clause BSD) is included in
the same source directory as the tiled.client.cache
module. (Cachey itself
is used in the server, where the use case is a better fit.)
|
A client-side cache of data from the server. |
|
An in-memory cache of data from the server |
|
An on-disk cache of data from the server |
|
Download a local cache for Tiled so access is fast and can work offline. |
|
Object to track scores of cache |
Context¶
|
Wrap an httpx.Client with an optional cache and authentication functionality. |
Accept a URI to a specific node. |
|
Construct a Context around a FastAPI app. |
|
See login. |
|
Execute refresh flow. |
|
Depending on the server's authentication method, this will prompt for username/password: |
|
Log out of the current session (if any). |
|
A view of the current access and refresh tokens. |