 | REST API
REST API |
The tap-rest-api-msdk extractor pulls data from REST API that can then be sent to a destination using a loader.
Getting Started
Prerequisites
If you haven't already, follow the initial steps of the Getting Started guide:
Installation and configuration
-
Add the tap-rest-api-msdk extractor to your project
using
:meltano add -
Configure the tap-rest-api-msdk settings using
:meltano config -
Test that extractor settings are valid using
:meltano config
meltano add extractor tap-rest-api-msdkmeltano config tap-rest-api-msdk set --interactivemeltano config tap-rest-api-msdk testNext steps
Follow the remaining steps of the Getting Started guide:
If you run into any issues, learn how to get help.
Capabilities
The current capabilities for
tap-rest-api-msdk
may have been automatically set when originally added to the Hub. Please review the
capabilities when using this extractor. If you find they are out of date, please
consider updating them by making a pull request to the YAML file that defines the
capabilities for this extractor.
This plugin has the following capabilities:
- state
- catalog
- discover
- about
- stream-maps
You can
override these capabilities or specify additional ones
in your meltano.yml by adding the capabilities key.
Settings
The
tap-rest-api-msdk settings that are known to Meltano are documented below. To quickly
find the setting you're looking for, click on any setting name from the list:
You can
override these settings or specify additional ones
in your meltano.yml by adding the settings key.
Please consider adding any settings you have defined locally to this definition on MeltanoHub by making a pull request to the YAML file that defines the settings for this plugin.
Name (name)
-
Environment variable:
TAP_REST_API_MSDK_NAME
(required) name of the stream.
API Base URL (api_url)
-
Environment variable:
TAP_REST_API_MSDK_API_URL
(required) the base url/endpoint for the desired api.
Path (path)
-
Environment variable:
TAP_REST_API_MSDK_PATH
(optional) the path appeneded to the api_url.
Params (params)
-
Environment variable:
TAP_REST_API_MSDK_PARAMS
(optional) an object of objects that provide the params in a requests.get method.
Headers (headers)
-
Environment variable:
TAP_REST_API_MSDK_HEADERS
(optional) an object of headers to pass into the api calls.
Records Path (records_path)
-
Environment variable:
TAP_REST_API_MSDK_RECORDS_PATH
(optional) a jsonpath string representing the path in the requests response that contains the records to process. Defaults to $[*].
Primary Keys (primary_keys)
-
Environment variable:
TAP_REST_API_MSDK_PRIMARY_KEYS
(required) a list of the json keys of the primary key for the stream.
Replication Key (replication_key)
-
Environment variable:
TAP_REST_API_MSDK_REPLICATION_KEY
(optional) the json key of the replication key. Note that this should be an incrementing integer or datetime object.
Except Keys (except_keys)
-
Environment variable:
TAP_REST_API_MSDK_EXCEPT_KEYS
This tap automatically flattens the entire json structure and builds keys based on the corresponding paths. Keys, whether composite or otherwise, listed in this dictionary will not be recursively flattened, but instead their values will be turned into a json string and processed in that format. This is also automatically done for any lists within the records; therefore, records are not duplicated for each item in lists.
Number Inference Records (num_inference_records)
-
Environment variable:
TAP_REST_API_MSDK_NUM_INFERENCE_RECORDS
(optional) number of records used to infer the stream's schema. Defaults to 50.
Something missing?
This page is generated from a YAML file that you can contribute changes to.
Edit it on GitHub!Looking for help?
#plugins-generalInstall
meltano add extractor tap-rest-api-msdk
Maintenance Status
-brightgreen){kind=icon}
Maintainer
Meltano Stats
-750-c0c0c4){kind=icon}
-57-c0c0c4){kind=icon}
PyPI Stats
Keywords
