flexmeasures.data.services.data_sources

Functions

flexmeasures.data.services.data_sources.add_and_flush_source(source: DataSource, query) → DataSource

Add and flush a new source, or return the identical source another transaction inserted since query looked for it.

Parameters:

  • source – the new data source, not yet added to the session.

  • query – the lookup that found no such source, to fetch the one another transaction inserted in the meantime.

flexmeasures.data.services.data_sources.find_duplicate_sources() → dict[int, int]

Find data sources identical to another, as the unique index on data sources defines them.

Such duplicates can only exist in a database that predates that index, whose migration merges each group into one source. Which one that is has to be decided the same way here, because this is what tells a host, before they upgrade, which of their waiting jobs name a source that is about to go: the source that recorded most recently keeps its ID. Recency is the belief time rather than the event a belief is about, since a generator mostly records about the future. A host naming a source with -x keep-source=<id> turns one group around, which this cannot know about.

Kept in step with SURVIVOR_RANKING in migration 7c4e1a9d2b58, which cannot import from here.

Returns:

the ID of each duplicate data source, mapped to the ID of the data source it will be merged into

flexmeasures.data.services.data_sources.find_referenced_source_ids(value) → set[int]

Find the data source IDs referred to in a JSON-like value, such as the arguments of a job.

Source IDs are recognised under the keys that hold them in sensor references, reporter inputs, and report job arguments.

flexmeasures.data.services.data_sources.first_source(query) → DataSource | None

Return the oldest data source that a lookup by (part of) a source’s identity finds, if any.

flexmeasures.data.services.data_sources.get_data_generator(source: DataSource | None, model: str, config: dict, save_config: bool, data_generator_type: Type[DG]) → DG | None
flexmeasures.data.services.data_sources.get_or_create_source(source: User | str, source_type: str | None = None, model: str | None = None, version: str | None = None, attributes: dict | None = None, account: Account | None = None, flush: bool = True) → DataSource
flexmeasures.data.services.data_sources.get_readable_source_account_ids() → list[int] | None

Return the ids of the accounts whose data sources the current user may read.

Returns None to say that every account’s sources are readable, which is what admin access amounts to.

flexmeasures.data.services.data_sources.get_source_or_none(source: int | str, source_type: str | None = None) → DataSource | None
Parameters:

  • source – source id

  • source_type – optionally, filter by source type

flexmeasures.data.services.data_sources.readable_source_filter() → ColumnElement[bool] | None

A condition selecting the data sources the current user may read.

On top of the sources which are theirs to work with (see usable_source_filter), this covers the sources which have recorded data on a sensor they may read, because “what computed this number?” is a fair question about data one is allowed to see.

Returns None to say that every source is readable, which is what admin access amounts to.

flexmeasures.data.services.data_sources.usable_source_filter() → ColumnElement[bool] | None

A condition selecting the data sources which are the current user’s to work with.

These are the sources they may reuse for an automation of their own, and the ones whose stored configuration they may read, which is a stronger thing to be allowed than reading what a source has computed: see readable_source_filter.

Returns None to say that every source qualifies, which is what admin access amounts to.

flexmeasures.data.services.data_sources.user_may_read_source(source: DataSource) → bool

Whether the current user may read the given data source.

See readable_source_filter for what makes a source readable.

flexmeasures.data.services.data_sources.user_may_use_source(source: DataSource) → bool

Whether the given data source is the current user’s to work with.

See usable_source_filter for what that means, and why it is a stronger thing to be allowed than reading the source.