PANOPTES Data tools#

Tools for searching and downloading PANOPTES data.

Install#

Install from pip:

pip install panoptes-data

Examples#

See the example Jupyter Notebooks in the notebooks/ directory.

Finding observations#

from panoptes.data.search import search_observations
from panoptes.data.observations import ObservationInfo

# Find some observations
results = search_observations(by_name='M42')

# Use last result entry to create ObservationInfo object.
obs_info = ObservationInfo(meta=results.iloc[0])
print(obs_info.meta)

# Create an ObservationInfo object directly from a sequence_id.
obs_info = ObservationInfo('PAN001_14d3bd_20180113T052325')
# But then there is no metadata:
print(obs_info.meta)

Sample output (truncated):

camera_id                                           14d3bd
camera_lens_serial_number                        HA0028608
camera_serial_number                           12070048413
coordinates_mount_dec                            -6.229778
coordinates_mount_ra                               76.0815
exptime                                              120.0
field_name                                         Wasp 35
num_images                                            28.0
sequence_id                  PAN001_14d3bd_20180113T052325
software_version                                POCSv0.6.0
time                             2018-01-13 05:23:25+00:00
total_exptime                                       3360.0
unit_id                                             PAN001
Name: 6121, dtype: object

Downloading images#

The ObservationInfo object makes it easy to download the files:

obs_info.download_images()

Command-line tools#

There is a simple command line tool that allows for both searching and downloading of images and metadata.

Search for observations:#

panoptes-data search --name M42 --min-num-images 90

Example table output:

| sequence_id                   | field_name   | unit_id   |   coordinates_mount_ra |   coordinates_mount_dec |   num_images |   exptime |   total_exptime | time                      |
|:------------------------------|:-------------|:----------|-----------------------:|------------------------:|-------------:|----------:|----------------:|:--------------------------|
| PAN022_977c86_20220108T090553 | M42          | PAN022    |                83.8221 |                -5.39111 |           95 |   90      |            8550 | 2022-01-08 09:05:53+00:00 |
| PAN022_538cc6_20220108T090553 | M42          | PAN022    |                83.8221 |                -5.39111 |           95 |   89      |            8455 | 2022-01-08 09:05:53+00:00 |
| PAN019_42433a_20220114T085722 | M42          | PAN019    |                83.8221 |                -5.39111 |           90 |   90      |            8100 | 2022-01-14 08:57:22+00:00 |
| PAN019_c623e9_20220114T085722 | M42          | PAN019    |                83.8221 |                -5.39111 |           90 |   89.0222 |            8012 | 2022-01-14 08:57:22+00:00 |
| PAN019_c623e9_20220115T082108 | M42          | PAN019    |                83.8221 |                -5.39111 |          105 |   89.019  |            9347 | 2022-01-15 08:21:08+00:00 |
| PAN019_42433a_20220115T082108 | M42          | PAN019    |                83.8221 |                -5.39111 |          105 |   90.0095 |            9451 | 2022-01-15 08:21:08+00:00 |

Downloading all images for an observation:#

panoptes-data download PAN022_977c86_20220108T090553

Get all metadata for a unit in a given date range:#

panoptes-data get-metadata --unit-id PAN022 --start-date '2022-01-08'

See panoptes-data --help for more options.

Building the documentation#

Follow these steps to build the HTML documentation locally.

Create and activate a virtual environment (recommended):

python3 -m venv .venv
. .venv/bin/activate

Upgrade packaging tools and install the docs requirements:

python -m pip install --upgrade pip setuptools wheel
pip install -r docs/requirements.txt

For more complete API docs (fewer autodoc import warnings), install the project runtime dependencies in the same environment. This will allow Sphinx to import modules like astropy and typer during the build:

# Install the package and its dependencies (editable install is convenient during development)
pip install -e .
# or install specific packages if you prefer:
# pip install astropy typer

Build the docs and open them locally:

cd docs
make html
# open the index in your browser
open _build/html/index.html

Notes

If you prefer not to install heavy runtime packages, docs/conf.py includes autodoc_mock_imports for some common heavy dependencies so the docs can still build; however installing the real packages gives more complete API documentation.
If you see autodoc import warnings mentioning other missing packages, install those packages into your .venv or add them to autodoc_mock_imports in docs/conf.py.

CI / Automated docs build#

For CI (for example GitHub Actions) it’s convenient to use a full docs requirements file that includes runtime packages needed for autodoc. A sample workflow is provided at .github/workflows/docs.yml.

Quick CI steps (what the workflow runs):

# create and activate a venv (CI runners already provide Python environment)
python3 -m venv .venv
. .venv/bin/activate

# install full docs deps
python -m pip install --upgrade pip setuptools wheel
pip install -r docs/requirements.txt

# install the package (editable) so autodoc can import it
pip install -e .

# build
cd docs
make html

The CI example workflow will upload the generated _build/html directory as an artifact so you can preview the generated site.

(See .github/workflows/docs.yml for the exact GitHub Actions configuration.)