nrgpy.cloud_api package

nrgpy.cloud_api package#

Submodules#

nrgpy.cloud_api.auth module#

class nrgpy.cloud_api.auth.CloudApi(client_id: str = '', client_secret: str = '', url_base: str = 'https://cloud-api.nrgsystems.com/nrgcloudcustomerapi/')[source]#

Bases: object

Parent class for NRG Cloud API functionality

nrgpy simplifies usage of the NRG Cloud APIs. See the documentation for the cloud_api.sites, .export, and .convert modules for more information.

For non-nrgpy implementations, the following Usage information may be helpful.

The base url for the NRG Cloud APIs is

https://cloud-api.nrgsystems.com/nrgcloudcustomerapi/

Use client ID and secret to obtain a bearer token at the /token endpoint. This token is good for 24 hours.

You will be limited to creating 10 tokens per day, though normally one token should suffice, so please cache. nrgpy’s cloud_api modules will automatically manage bearer tokens, refreshing only when necessary.

Endpoints:

base: https://cloud-api.nrgsystems.com/nrgcloudcustomerapi/

/token: for accessing bearer token. Client ID and Secret required.
/sites: Get list of sites. Bearer token required
/convert: Convert RLD file to TXT. Bearer token required
/export: Export TXT or RLD files for a given date range. NEC files may be used to format TXT outputs. Bearer token required.

load_token() → None[source]#: read session token from pickle file

maintain_session_token() → None[source]#: maintain a current/valid session token for data service api

prepare_file_bytes(filename: str = '') → bytes[source]#

request_session_token() → None[source]#

Generates a new session token for convert service api

Requires an active account with NRG Cloud. To sign up for a free account, go to:

https://cloud.nrgsystems.com

Client ID and Secret can be accessed here:

https://cloud.nrgsystems.com/data-manager/api-setup

Parameters:

client_idstr: obtained from NRG Systems
client_secretstr

Returns:

session_tokenstr: valid for 24 hour
session_start_timedatetime: start time of 24 hour countdown

save_token() → None[source]#: save session token in token pickle file

token_valid() → bool[source]#

check if token is still valid

Parameters:

session_start_timedatetime: generated at time of token request

Returns:

statusbool: true if still valid, false if expired

nrgpy.cloud_api.auth.cloud_api#: alias of CloudApi

nrgpy.cloud_api.auth.is_authorized(resp) → bool[source]#

nrgpy.cloud_api.convert module#

class nrgpy.cloud_api.convert.CloudConvert(rld_dir: str = '', out_dir: str = '', filename: str = '', site_filter: str = '', filter2: str = '', start_date: str = '1970-01-01', end_date: str = '2150-12-31', client_id: str = '', client_secret: str = '', url_base: str = 'https://cloud-api.nrgsystems.com/nrgcloudcustomerapi/', export_type: str = 'measurements', nec_file: str = '', unzip: bool = True, progress_bar: bool = True, **kwargs)[source]#

Bases: CloudApi

Uses NRG hosted web-based API to convert RLD and RWD files to text format To sign up for the service, go to https://cloud.nrgsystems.com/.

Note that the site must exist in the NRG Cloud platform, and you must have Contributor or Administrator level access to the site to use these features.

Examples

Convert a single raw data file to Text with NRG Convert API

>>> import nrgpy
>>> filename = "/home/user/data/sympro/000123/000123_2019-05-23_19.00_003672.rld
>>> client_id = "go to https://cloud.nrgsystems.com/data-manager/api-setup for access"
>>> client_secret = "go to https://cloud.nrgsystems.com/data-manager/api-setup for access"
>>> converter = nrgpy.CloudConvert(
        file_filter=file_filter,
        filename=filename,
        client_id=client_id,
        client_secret=client_secret,
    )

Convert a folder of raw data files to Text with NRG Convert API

>>> import nrgpy
>>> file_filter = "000175"
>>> rld_directory = "rlds"
>>> txt_dir = "/home/user/data/sympro/000123/txt/"
>>> client_id = "go to https://cloud.nrgsystems.com/data-manager/api-setup for access"
>>> client_secret = "go to https://cloud.nrgsystems.com/data-manager/api-setup for access"
>>> converter = nrgpy.CloudConvert(
        file_filter=file_filter,
        rld_dir=rld_directory,
        out_dir=txt_dir,
        client_id=client_id,
        client_secret=client_secret,
        start_date="2020-01-01",
        end_date="2020-01-31",
    )
>>> converter.process()

Attributes:

rld_dirstr (path-like): path to rld file directory
out_dirstr (path-like): path to save text export files
filenamestr: provide for single file conversion
site_filterstr, optional: text filter for limiting file set
filter2str, optional: another text filter…
start_datestr, optional: text start date to filter on “YYYY-mm-dd”
end_datestr, optional: text end date to filter on “YYYY-mm-dd”
client_idstr: provided by NRG Systems
client_secretstr: provided by NRG Systems
nec_filestr, optional: path to NEC file for custom export formatting
export_typestr: [measurements], diagnostic, events, communication
unzipbool: whether to extract the .txt data file from the .zip file
session_tokenstr
headersstr: headers passed in API call
datastr: data passed in API call
respstr: API response
export_filepathstr (path-like): path of export file

process() → None[source]#

single_file(rld) → None[source]#

nrgpy.cloud_api.convert.cloud_convert#: alias of CloudConvert

nrgpy.cloud_api.export module#

class nrgpy.cloud_api.export.CloudExport(out_dir: str = '', site_id: str = '', site_number: str = '', logger_sn: str = '', start_date: str = '2014-01-01', end_date: str = '2030-12-31', file_format: str = 'txt', client_id: str = '', client_secret: str = '', url_base: str = 'https://cloud-api.nrgsystems.com/nrgcloudcustomerapi/', nec_file: str = '', export_type: str = 'measurements', interval: str = '', concatenate: bool = True, unzip: bool = True, **kwargs)[source]#

Bases: CloudApi

Uses NRG hosted web-based API to download data in text format To sign up for the service, go to https://cloud.nrgsystems.com

Note that the site must exist in the NRG Cloud platform, and you must have Contributor or Administrator level access to the site to use these features.

Use the Site Number or NRG Cloud Site ID to choose the site.

Examples

Download 15 days of data with an NEC file applied, and read data

>>> import nrgpy
>>>
>>> client_id = "go to https://cloud.nrgsystems.com for access"
>>> client_secret = "go to https://cloud.nrgsystems.com for access"
>>> save_dir = '/path/to/exported/data'
>>>
>>> exporter = nrgpy.CloudExport(
        client_id=client_id,
        client_secret=client_secret,
        out_dir=save_dir,
        nec_file='12vbat.nec',
        site_id=245,
        start_date="2021-05-01",
        end_date="2021-05-15",
    )
>>> exporter.export()
>>> # read result
>>> reader = nrgpy.sympro_txt_read(exporter.export_filepath)
>>> reader.format_site_data()
>>> if reader:
>>>     print(f"Site number               : {reader.site_number}")
>>>     print(f"Site description          : {reader.site_description}")
>>>     reader.interval_check = nrgpy.check_intervals(reader.data)
>>> else:
>>>     print("unable to get reader")

Download 15 days of data files from ZX 300 site

>>> import nrgpy
>>>
>>> client_id = "go to https://cloud.nrgsystems.com for access"
>>> client_secret = "go to https://cloud.nrgsystems.com for access"
>>> save_dir = '/path/to/exported/data'
>>> file_format = 'zx'
>>>
>>> exporter = nrgpy.CloudExport(
        client_id=client_id,
        client_secret=client_secret,
        out_dir=save_dir,
        file_format='zx',
        site_id=57,
        start_date="2022-05-01",
        end_date="2022-05-15",
    )
>>> exporter.export()

Attributes:

out_dirstr (path-like): path to save exported data
site_idint: NRG Cloud site identifier (NOT the site number)
site_numberint: site number
logger_snint: serial number of data logger
start_datestr (‘{YYYY}-{MM}-{DD}T{HH}:{MM}:{SS}’): start date/time of data export
end_datestr (‘{YYYY}-{MM}-{DD}T{HH}:{MM}:{SS}’): end date/time of data export
file_format{‘txt’, ‘rld’, ‘zx’}: whether SymPRO tab-delimited text or binary output or ZX
client_idstr: available in the NRG Cloud portal
client_secretstr: available in the NRG Cloud portal
nec_filestr, optional: path to NEC file for custom export formatting
export_type{‘measurements’, ‘diagnostic’, ‘events’, ‘communication’}: which type of data to export
interval{‘oneMinute’, ‘twoMinute’, ‘fiveMinute’, ‘tenMinute’, ‘fifteenMinute’,: ‘thirtyMinute’, ‘Hour’, ‘Day’}, optional averaging interval of exported data; must be a multiple of the logger’s statistical interval
unzipbool: whether to extract the .txt data file from the .zip file
session_tokenstr
headersstr: headers passed in API call
datastr: data passed in API call
respstr: API response
export_filepathstr (path-like): path of export file

export() → None[source]#: Export data using the NRG Cloud API.

format_data_for_api_versions() → None[source]#

prepare_post_data() → None[source]#

process_zip() → None[source]#

nrgpy.cloud_api.export.cloud_export#: alias of CloudExport

nrgpy.cloud_api.jobs module#

class nrgpy.cloud_api.jobs.CloudExportJob(out_dir: str = '', site_id: str = '', site_number: str = '', logger_sn: str = '', start_date: str = '2014-01-01', end_date: str = '2023-12-31', file_format: str = 'txt', client_id: str = '', client_secret: str = '', url_base: str = 'https://cloud-api.nrgsystems.com/nrgcloudcustomerapi/', nec_file: str = '', export_type: str = 'measurements', concatenate: bool = True, interval: str = '', unzip: bool = True, **kwargs)[source]#

Bases: CloudExport

Uses NRG hosted web-based API to download data in text format as an export job To sign up for the service, go to https://cloud.nrgsystems.com

Note that the site must exist in the NRG Cloud platform, and you must have Contributor or Administrator level access to the site to use these features.

Use for creating exports of any size. Extends nrgpy.cloud_export class

Parameters:

out_dirstr (path-like): path to save exported data
site_idint: NRG Cloud site identifier (NOT the site number)
site_numberint: site number
logger_snstr or int: serial number of data logger (like, 820612345)
start_datestr: “YYYY-MM-DD HH:MM:SS” format, if just date it will return the whole day times are in logger local time
end_datestr: “YYYY-MM-DD HH:MM:SS” format, if just date it will return the whole day times are in logger local time
client_idstr: available in the NRG Cloud portal
client_secretstr: available in the NRG Cloud portal
nec_filestr, optional: path to NEC file for custom export formatting
export_typestr: [‘measurements’], ‘diagnostic’, ‘events’, ‘communication’
file_formatstr: [‘txt’], ‘rld’, ‘zx’ - whether tab-delimited text or binary output
intervalstr, optional: ‘oneMinute’, ‘twoMinute’, ‘fiveMinute’, ‘tenMinute’, ‘fifteenMinute’, ‘thirtyMinute’, ‘Hour’, ‘Day’ must be a multiple of the logger’s statistical interval. if not specified, will use “native” interval (usually oneMinute for solar, tenMinute for wind)
concatenatebool: (True) set to False to return original CSV files in export (ZX only)
unzipbool: (True) whether to extract the .txt data file from the .zip file

Returns:

object: export object, including API response

Examples

Download one year of data with an NEC file applied, and read data

>>> import nrgpy
>>>
>>> client_id = "go to https://cloud.nrgsystems.com for access"
>>> client_secret = "go to https://cloud.nrgsystems.com for access"
>>> save_dir = '/path/to/exported/data'
>>>
>>> exporter = nrgpy.CloudExportJob(
        client_id=client_id,
        client_secret=client_secret,
        out_dir=save_dir,
        nec_file='12vbat.nec',
        site_number=3456,
        start_date="2021-04-01",
        end_date="2022-03-31",
        file_format="singleFile",  # use "multipleFiles" for RLD export
        unzip=True,
    )
>>>
>>> exporter.create_export_job()
>>> exporter.monitor_export_job(download=True)
>>> # read resulting files
>>> reader = nrgpy.sympro_txt_read(exporter.export_filepath)
>>> reader.format_site_data()
>>> if reader:
>>>     print(f"Site number               : {reader.site_number}")
>>>     print(f"Site description          : {reader.site_description}")
>>>     reader.interval_check = nrgpy.check_intervals(reader.data)
>>> else:
>>>     print("unable to get reader")

Download 15 days of data files from ZX 300 lidar site

>>> import nrgpy
>>>
>>> client_id = "go to https://cloud.nrgsystems.com for access"
>>> client_secret = "go to https://cloud.nrgsystems.com for access"
>>> save_dir = '/path/to/exported/data'
>>>
>>> exporter = nrgpy.CloudExportJob(
        client_id=client_id,
        client_secret=client_secret,
        out_dir=save_dir,
        site_number=567,
        start_date="2022-09-01",
        end_date="2022-09-15",
        file_format="zx",
        concatenate=True,
        unzip=True,
    )
>>>
>>> exporter.create_export_job()
>>> exporter.monitor_export_job(download=True)

check_export_job() → None[source]#

Checks the status of an export job

Uses self.job_id as reference

create_export_job() → None[source]#: Create export job

download_export() → None[source]#

monitor_export_job(download: bool = False) → None[source]#

Monitor the status of an export job

Uses self.job_id as reference

nrgpy.cloud_api.jobs.export_job#: alias of CloudExportJob

nrgpy.cloud_api.sites module#

class nrgpy.cloud_api.sites.CloudSites(client_id: str, client_secret: str, url_base: str = 'https://cloud-api.nrgsystems.com/nrgcloudcustomerapi/')[source]#

Bases: CloudApi

Find NRG Cloud sites associated with a user.

Examples

>>> import nrgpy
>>> client_id = 'go to https://cloud.nrgsystems.com for access'
>>> client_secret = 'go to https://cloud.nrgsystems.com for access'
>>> sites = nrgpy.CloudSites(client_id=client_id, client_secret=client_secret)
>>> sites.sites_df
siteId  siteNumber    siteDescription               project  loggerSerialNumber
0      33        6716           SunnyDog              SunnyDog           820606716
1     213        1234    Suntastic South             Suntastic           820601234
>>> sites.get_siteid(site_number=6716)
33

Attributes:

client_idstr: available in the NRG Cloud portal
client_secretstr: available in the NRG Cloud portal
sites_listlist of dict
sites_dfpandas dataframe

get_siteid(site_number: str = '', logger_sn: str = '') → None[source]#

Get NRG Cloud site ID that corresponds to site number and/or logger SN

Parameters:

site_numberint: site number
logger_snint: serial number of logger (beginning with 8206)

Returns:

int: corresponding site ID

get_sites() → None[source]#: Retrieve list of sites user has access to.

nrgpy.cloud_api.sites.cloud_sites#: alias of CloudSites

nrgpy.cloud_api.upload module#

class nrgpy.cloud_api.upload.CloudImport(in_dir: str = '', filename: str = '', site_filter: str = '', filter2: str = '', start_date: str = '1970-01-01', end_date: str = '2150-12-31', client_id: str = '', client_secret: str = '', url_base: str = 'https://cloud-api.nrgsystems.com/nrgcloudcustomerapi/', progress_bar: bool = True, **kwargs)[source]#

Bases: CloudApi

Uses NRG hosted web-based API to import RLD and CSV/CSV.zip files to existing sites in NRG Cloud. To sign up for the service, go to https://cloud.nrgsystems.com/.

Note that the site must exist in the NRG Cloud platform, and you must have Contributor or Administrator level access to the site to use these features.

Examples

Import a single raw data file with NRG Import API

>>> import nrgpy
>>> filename = "/home/user/data/sympro/000123/000123_2019-05-23_19.00_003672.filename
>>> client_id = "go to https://cloud.nrgsystems.com/data-manager/api-setup for access"
>>> client_secret = "go to https://cloud.nrgsystems.com/data-manager/api-setup for access"
>>> importer = nrgpy.CloudImport(
        filename=filename,
        client_id=client_id,
        client_secret=client_secret,
    )

Import a folder of data files with NRG Import API

>>> import nrgpy
>>> file_filter = "000175"
>>> in_directory = "filenames"
>>> client_id = "go to https://cloud.nrgsystems.com/data-manager/api-setup for access"
>>> client_secret = "go to https://cloud.nrgsystems.com/data-manager/api-setup for access"
>>> importer = nrgpy.CloudImport(
        file_filter=file_filter,
        in_dir=in_directory,
        client_id=client_id,
        client_secret=client_secret,
        start_date="2020-01-01",
        end_date="2020-01-31",
    )
>>> importer.process()

Attributes:

in_dirstr (path-like): path to data file directory
filenamestr: provide for single file import
site_filterstr, optional: text filter for limiting file set
filter2str, optional: another text filter…
start_datestr, optional: text start date to filter on “YYYY-mm-dd”
end_datestr, optional: text end date to filter on “YYYY-mm-dd”
client_idstr: provided by NRG Systems
client_secretstr: provided by NRG Systems
session_tokenstr
headersstr: headers passed in API call
datastr: data passed in API call
respstr: API response
job_idsdict: dictionary of filenames and job ids

get_valid_files_from_in_dir() → list[source]#

process()[source]#

single_file(filename: str = '')[source]#

nrgpy.cloud_api.upload.cloud_import#: alias of CloudImport

nrgpy.cloud_api package

Contents

nrgpy.cloud_api package#

Submodules#

nrgpy.cloud_api.auth module#

nrgpy.cloud_api.convert module#

nrgpy.cloud_api.export module#

nrgpy.cloud_api.jobs module#

nrgpy.cloud_api.sites module#

nrgpy.cloud_api.upload module#

Module contents#