| Title: | Investment Data Access and Preparation Toolkit |
|---|---|
| Description: | A toolkit for loading, transforming, and managing structured investment-related data. Includes functions for retrieving macroeconomic and public financial series, regulatory filings, futures positioning, energy and fiscal data, market open-high-low-close-volume data, fund holdings, narrative feeds, and crypto derivatives from multiple providers. It detects time-series gaps and synchronizes local '.rds' data stores with sidecar metadata for analysis and trading workflows. It also generates structured dataset descriptions for large language model (LLM)-based analyst agents that produce R code for analysis and visualization. |
| Authors: | Oliver Zhou [aut, cre] |
| Maintainer: | Oliver Zhou <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 0.1.6 |
| Built: | 2026-10-07 13:08:30 UTC |
| Source: | https://github.com/oliverlds/investdatar |
Add Or Update One FRED Registry Entry
add_fred_registry_series( series_id, main_group = NULL, registry_path = get_fred_registry_file_path(), config = NULL )add_fred_registry_series( series_id, main_group = NULL, registry_path = get_fred_registry_file_path(), config = NULL )
series_id |
FRED series identifier. |
main_group |
Optional grouping label. If |
registry_path |
Optional registry JSON path. |
config |
Optional FRED API config. |
The added or updated row as a data.table.
Add Or Update An SEC Registry Company
add_sec_registry_company( ticker, cik = NULL, company_name = NULL, forms = c("10-K", "10-Q", "8-K"), concepts = NULL, active = TRUE, registry_path = get_sec_registry_file_path(), config = NULL )add_sec_registry_company( ticker, cik = NULL, company_name = NULL, forms = c("10-K", "10-Q", "8-K"), concepts = NULL, active = TRUE, registry_path = get_sec_registry_file_path(), config = NULL )
ticker |
Security ticker. |
cik |
Optional CIK; resolved from the SEC ticker mapping when omitted. |
company_name |
Optional company name. |
forms |
Optional filing forms retained by submissions sync. |
concepts |
Optional XBRL concepts retained by Company Facts sync. |
active |
Logical registry flag. |
registry_path |
Optional registry path. |
config |
Optional SEC configuration. |
The stored registry row.
Add Or Update One World Bank Registry Entry
add_wbstats_registry_series( indicator, country = NULL, freq = "Y", main_group = NA_character_, label = NA_character_, notes = NA_character_, active = TRUE, registry_path = get_wbstats_registry_file_path() )add_wbstats_registry_series( indicator, country = NULL, freq = "Y", main_group = NA_character_, label = NA_character_, notes = NA_character_, active = TRUE, registry_path = get_wbstats_registry_file_path() )
indicator |
World Bank indicator code. |
country |
Country or aggregate code. |
freq |
Frequency code. |
main_group |
Optional grouping label. |
label |
Optional display label. |
notes |
Optional free-text notes. |
active |
Logical flag stored in the registry. |
registry_path |
Optional registry JSON path. |
The added or updated row as a data.table.
Read the local Yahoo Finance cache for 000300.SS, compare its common daily
coverage with a fresh Eastmoney request, and write versioned JSON and CSV
audit artifacts. This function is read-only with respect to cached OHLC and
metadata files: it does not synchronize, repair, or overwrite market data.
audit_csi300_cache_integrity( local_path = NULL, output_dir = NULL, from = NULL, to = NULL, close_tolerance = 0.01, baostock_corroboration = NULL )audit_csi300_cache_integrity( local_path = NULL, output_dir = NULL, from = NULL, to = NULL, close_tolerance = 0.01, baostock_corroboration = NULL )
local_path |
Yahoo Finance cache directory. Defaults to the configured Yahoo Finance data path. |
output_dir |
Directory for audit artifacts. Defaults to
|
from |
Optional inclusive start date for cached rows under review. |
to |
Optional inclusive end date for cached rows under review. |
close_tolerance |
Non-negative absolute close-price tolerance in index points after both closes are normalized to Eastmoney's published two-decimal quote precision. Rows strictly above this value are discrepancies. |
baostock_corroboration |
Optional named list of independently obtained BaoStock comparison evidence. It is recorded only as corroboration. |
baostock_corroboration is optional external evidence recorded verbatim in
the JSON report. It never triggers a BaoStock request and does not add a
BaoStock runtime dependency.
A list containing an audit summary, discrepancy data.table, and
JSON and CSV artifact paths.
Compare completed daily rows in catalog-backed Yahoo Finance caches with a
fresh Yahoo overlap. The audit writes versioned JSON and CSV artifacts only;
it never synchronizes, repairs, or modifies a cache. Registry symbols without
catalog metadata produce explicit skipped_missing_catalog_metadata findings.
audit_yahoofinance_recent_overlap( registry = get_yahoofinance_registry(), local_path = NULL, output_dir = NULL, overlap_days = 35L, as_of = Sys.time(), fallback_corroboration = c("on_issue", "none", "all"), require_catalog_metadata = TRUE, max_attempts = 3L, retry_delay_seconds = 1 )audit_yahoofinance_recent_overlap( registry = get_yahoofinance_registry(), local_path = NULL, output_dir = NULL, overlap_days = 35L, as_of = Sys.time(), fallback_corroboration = c("on_issue", "none", "all"), require_catalog_metadata = TRUE, max_attempts = 3L, retry_delay_seconds = 1 )
registry |
Yahoo Finance registry table. Defaults to the configured runtime registry. |
local_path |
Yahoo Finance cache directory. Defaults to the configured Yahoo Finance data path. |
output_dir |
Audit artifact directory. Defaults to
|
overlap_days |
Number of recent calendar days to inspect. Defaults to
|
as_of |
UTC time used to exclude the current market-calendar date.
For |
fallback_corroboration |
One of |
require_catalog_metadata |
When |
max_attempts |
Maximum Yahoo attempts per catalog-backed instrument. |
retry_delay_seconds |
Initial retry delay in seconds. |
A list with summary, findings, and JSON/CSV artifact_paths.
Create an absent runtime registry from the tracked package seed. Existing registries are never overwritten; they are validated and an actionable error is raised if required fallback declarations have drifted.
bootstrap_yahoofinance_registry( registry_path = get_yahoofinance_registry_file_path(), seed_path = get_yahoofinance_seed_registry_path() )bootstrap_yahoofinance_registry( registry_path = get_yahoofinance_registry_file_path(), seed_path = get_yahoofinance_seed_registry_path() )
registry_path |
Runtime registry JSON path. |
seed_path |
Package seed registry JSON path. |
Invisibly returns the runtime registry path.
Fetches AlphaVantage daily OHLCV data and creates a compact narrative for the resulting standardized market table.
describe_alphavantage_data(symbol, mode = c("compact", "full"), config = NULL)describe_alphavantage_data(symbol, mode = c("compact", "full"), config = NULL)
symbol |
Market symbol. |
mode |
Output size passed to AlphaVantage. |
config |
Optional AlphaVantage API configuration. |
Character scalar narrative.
Describe Local BEA Data
describe_bea_data(series_id, local_path = NULL)describe_bea_data(series_id, local_path = NULL)
series_id |
Registered series identifier. |
local_path |
Optional BEA storage directory. |
Character scalar narrative.
Fetches Binance OHLCV data and creates a compact narrative for the resulting standardized market table.
describe_binance_data( symbol = "ETHUSDT", interval = "1m", start_time = NULL, end_time = NULL, limit = 1500L, tz = "UTC", paginate = TRUE )describe_binance_data( symbol = "ETHUSDT", interval = "1m", start_time = NULL, end_time = NULL, limit = 1500L, tz = "UTC", paginate = TRUE )
symbol |
Trading pair symbol. |
interval |
Candlestick interval. |
start_time |
Optional start time. |
end_time |
Optional end time. |
limit |
Page size. Binance futures allows up to 1500. |
tz |
Time zone applied to returned timestamps. |
paginate |
Logical. Request multiple pages when needed. |
Character scalar narrative.
Describe Local BLS Data
describe_bls_data(series_id, local_path = NULL)describe_bls_data(series_id, local_path = NULL)
series_id |
BLS series identifier. |
local_path |
Optional BLS storage directory. |
Character scalar narrative.
Describe Local Census Data
describe_census_data(series_id, local_path = NULL)describe_census_data(series_id, local_path = NULL)
series_id |
Registered series identifier. |
local_path |
Optional Census storage directory. |
Character scalar narrative.
Describe Local CFTC COT Data
describe_cftc_cot_data(report_id, local_path = NULL)describe_cftc_cot_data(report_id, local_path = NULL)
report_id |
Registry report identifier. |
local_path |
Optional CFTC storage directory. |
Character scalar narrative.
Describe Local Crypto Derivatives Data
describe_crypto_derivatives( provider, dataset_type, symbol, interval = NULL, local_path = NULL )describe_crypto_derivatives( provider, dataset_type, symbol, interval = NULL, local_path = NULL )
provider |
|
dataset_type |
Funding, open-interest, mark/index price, basis, or Binance long-short-ratio dataset identifier. |
symbol |
Provider instrument identifier. |
interval |
Required Binance open-interest period; otherwise a local label. |
local_path |
Optional derivatives storage directory. |
Character scalar narrative.
Describe Local EIA Data
describe_eia_data(series_id, local_path = NULL)describe_eia_data(series_id, local_path = NULL)
series_id |
EIA series identifier. |
local_path |
Optional EIA storage directory. |
Character scalar narrative.
Describe Local Treasury Fiscal Data
describe_fiscaldata(dataset_id, local_path = NULL)describe_fiscaldata(dataset_id, local_path = NULL)
dataset_id |
Registry dataset identifier. |
local_path |
Optional Fiscal Data storage directory. |
Character scalar narrative.
Creates a compact narrative describing the structure and meaning of a locally stored FRED series for downstream analyst agents.
describe_fred_data( series_id, local_path = NULL, registry_path = get_fred_registry_file_path() )describe_fred_data( series_id, local_path = NULL, registry_path = get_fred_registry_file_path() )
series_id |
FRED series identifier. |
local_path |
Optional local storage path used to read the local series. |
registry_path |
Optional registry path used to resolve series metadata. |
Character scalar narrative.
Creates a compact narrative for standardized market OHLCV data.
describe_market_ohlcv_data(data, source_label = NULL)describe_market_ohlcv_data(data, source_label = NULL)
data |
A standardized OHLCV |
source_label |
Optional source label override. |
Character scalar narrative.
Creates a compact narrative for locally stored OKX candle data.
describe_okx_candle_data(inst_id, bar, local_path = NULL)describe_okx_candle_data(inst_id, bar, local_path = NULL)
inst_id |
Instrument identifier. |
bar |
Candle interval. |
local_path |
Optional local storage path. |
Character scalar narrative.
Fetches market OHLCV data through quantmod and creates a compact narrative
for the resulting standardized market table.
describe_quantmod_data( ticker, label = ticker, from = NULL, to = NULL, src = "yahoo", local_path = NULL )describe_quantmod_data( ticker, label = ticker, from = NULL, to = NULL, src = "yahoo", local_path = NULL )
ticker |
Market symbol passed to |
label |
Optional label to store in the standardized |
from |
Start date. If |
to |
End date. If |
src |
quantmod source, default |
local_path |
Optional local storage path used when |
Character scalar narrative.
Creates a compact narrative for locally stored RSS narrative data.
describe_rss_data(feed_id, local_path = NULL)describe_rss_data(feed_id, local_path = NULL)
feed_id |
Local feed identifier. |
local_path |
Optional local storage path. |
Character scalar narrative.
Describe Local SDMX Data
describe_sdmx_data(series_id, local_path = NULL)describe_sdmx_data(series_id, local_path = NULL)
series_id |
Registry series identifier. |
local_path |
Optional SDMX storage directory. |
Character scalar narrative.
Describe Local SEC Company Facts
describe_sec_companyfacts(cik, local_path = NULL)describe_sec_companyfacts(cik, local_path = NULL)
cik |
SEC Central Index Key. |
local_path |
Optional SEC Company Facts storage directory. |
Character scalar narrative.
Describe A Local SEC XBRL Frame
describe_sec_frame(taxonomy, tag, unit, period, local_path = NULL)describe_sec_frame(taxonomy, tag, unit, period, local_path = NULL)
taxonomy |
XBRL taxonomy, such as |
tag |
XBRL concept tag. |
unit |
XBRL unit, such as |
period |
SEC frame period, such as |
local_path |
Optional SEC frame cache directory. |
Character scalar narrative.
Describe Local SEC Submissions
describe_sec_submissions(cik, local_path = NULL)describe_sec_submissions(cik, local_path = NULL)
cik |
SEC Central Index Key. |
local_path |
Optional SEC submissions storage directory. |
Character scalar narrative.
Creates a compact narrative for locally stored Treasury raw rates.
describe_treasury_rates(dataset, local_path = NULL)describe_treasury_rates(dataset, local_path = NULL)
dataset |
Treasury dataset key. |
local_path |
Optional local storage path. |
Character scalar narrative.
Creates a compact narrative for locally stored World Bank indicator data.
describe_wbstats_data(indicator, country, freq = "Y", local_path)describe_wbstats_data(indicator, country, freq = "Y", local_path)
indicator |
World Bank indicator code. |
country |
Country or aggregate code. |
freq |
Frequency code. |
local_path |
Local storage path. |
Character scalar narrative.
Detects gaps in a local time series using either a fixed or calendar frequency.
detect_time_gaps(dt, time_col, frequency, tolerance = 1e-04, tz = "UTC")detect_time_gaps(dt, time_col, frequency, tolerance = 1e-04, tz = "UTC")
dt |
A data.table-compatible object. |
time_col |
Name of the time column. |
frequency |
Frequency string or candle interval. |
tolerance |
Numeric tolerance applied to fixed-interval gaps. |
tz |
Time zone used when coercing timestamps. |
A data.table describing detected gaps.
Detect Gaps In Local FRED Data
detect_time_gaps_fred(x, frequency = NULL, local_path = NULL)detect_time_gaps_fred(x, frequency = NULL, local_path = NULL)
x |
A FRED |
frequency |
Optional explicit frequency string. |
local_path |
Optional local storage path if |
A data.table of gaps.
Detect Time Gaps In OKX Candle Data
detect_time_gaps_okx_candle(dt, bar = "4H", tolerance = 1e-04)detect_time_gaps_okx_candle(dt, bar = "4H", tolerance = 1e-04)
dt |
A candle |
bar |
Candle interval. |
tolerance |
Numeric tolerance for fixed-width gap detection. |
A data.table.
Detect Gaps In Local World Bank Data
detect_time_gaps_wbstats( x = NULL, indicator = NULL, country = NULL, freq = "Y", local_path = NULL )detect_time_gaps_wbstats( x = NULL, indicator = NULL, country = NULL, freq = "Y", local_path = NULL )
x |
A World Bank |
indicator |
Optional indicator code when reading local data. |
country |
Optional country code when reading local data. |
freq |
Frequency code. |
local_path |
Optional local storage path. |
A data.table of gaps.
Compares the read-only local CNH=X cache with a fresh TLS-verified
Eastmoney 133.USDCNH pull. The resulting JSON and CSV artifacts contain
field-level differences, source timestamps, trading-date labels, provenance,
tolerance calculations, and conservative classifications. This function
never modifies cache or synchronization files and does not create repair
candidates.
diagnose_cnh_cache_integrity( local_path = NULL, output_dir = NULL, from = NULL, to = NULL, overlap_days = 35L )diagnose_cnh_cache_integrity( local_path = NULL, output_dir = NULL, from = NULL, to = NULL, overlap_days = 35L )
local_path |
Yahoo Finance cache directory. Defaults to the configured Yahoo Finance data path. |
output_dir |
Directory for diagnostic artifacts. Defaults to
|
from |
Optional inclusive date. Defaults to the recent cache overlap. |
to |
Optional inclusive date. Defaults to the recent cache overlap. |
overlap_days |
Number of calendar days used when |
The Yahoo and Eastmoney identifiers are both interpreted as USD/CNH, meaning
CNH units per one USD. Yahoo's CNH=X is retained as the provider symbol;
Eastmoney's 133.USDCNH is the corresponding provider symbol. Eastmoney
daily labels are reported with the provider's Asia/Shanghai market-date
convention; its HTTP response does not expose a bar timezone explicitly.
A list with summary, field-level differences, and artifact paths.
Returns a standardized OHLCV data.table with common market-schema columns:
source, symbol, interval, datetime, date, open, high, low,
close, and volume.
fetch_quantmod_OHLC( ticker, label = ticker, from, to, src = "yahoo", raw_data = FALSE, max_attempts = 3L, retry_delay_seconds = 1, fallback_source = NULL, fallback_ticker = ticker, require_start_coverage = FALSE )fetch_quantmod_OHLC( ticker, label = ticker, from, to, src = "yahoo", raw_data = FALSE, max_attempts = 3L, retry_delay_seconds = 1, fallback_source = NULL, fallback_ticker = ticker, require_start_coverage = FALSE )
ticker |
Market symbol passed to |
label |
Optional label to store in the standardized |
from |
Start date. |
to |
End date. |
src |
quantmod source, default |
raw_data |
Logical. If |
max_attempts |
Maximum bounded attempts for a transient source failure. |
retry_delay_seconds |
Initial retry delay in seconds; delays use exponential backoff. |
fallback_source |
Optional explicitly configured fallback provider.
Currently supports |
fallback_ticker |
Optional provider-specific fallback identifier. |
require_start_coverage |
Logical. Require material coverage from |
A row is usable only if open, high, low, and close are finite. Isolated invalid rows are discarded. A window is materially incomplete when its end is more than seven calendar days behind the requested end, or, for an instrument with valid local history, when its start is more than seven calendar days late or fewer than half of the requested weekdays are present. The start rule is not applied to a newly listed instrument, and the calendar-day grace prevents weekend and market-holiday false positives.
data.table or raw xts object when raw_data = TRUE.
Get Alpha Vantage Series Registry
get_alphavantage_registry( registry_path = get_alphavantage_registry_file_path() )get_alphavantage_registry( registry_path = get_alphavantage_registry_file_path() )
registry_path |
Optional registry JSON path. |
A registry data.table.
Get Alpha Vantage Registry File Path
get_alphavantage_registry_file_path(config_dir = NULL)get_alphavantage_registry_file_path(config_dir = NULL)
config_dir |
Optional configuration directory used for fallback. |
Character scalar path.
Get BEA Series Registry
get_bea_registry(registry_path = get_bea_registry_file_path())get_bea_registry(registry_path = get_bea_registry_file_path())
registry_path |
Optional registry path. |
A registry data.table.
Get BEA Registry File Path
get_bea_registry_file_path(config_dir = NULL)get_bea_registry_file_path(config_dir = NULL)
config_dir |
Optional configuration directory. |
Character scalar path.
Get BLS Registry
get_bls_registry(registry_path = get_bls_registry_file_path())get_bls_registry(registry_path = get_bls_registry_file_path())
registry_path |
Optional registry path. |
A registry data.table.
Get BLS Registry File Path
get_bls_registry_file_path(config_dir = NULL)get_bls_registry_file_path(config_dir = NULL)
config_dir |
Optional configuration directory. |
Character scalar path.
Get Census Series Registry
get_census_registry(registry_path = get_census_registry_file_path())get_census_registry(registry_path = get_census_registry_file_path())
registry_path |
Optional registry path. |
A registry data.table.
Get Census Registry File Path
get_census_registry_file_path(config_dir = NULL)get_census_registry_file_path(config_dir = NULL)
config_dir |
Optional configuration directory. |
Character scalar path.
Get CFTC COT Registry
get_cftc_cot_registry(registry_path = get_cftc_cot_registry_file_path())get_cftc_cot_registry(registry_path = get_cftc_cot_registry_file_path())
registry_path |
Optional JSON registry path. |
A registry data.table.
Get CFTC COT Registry File Path
get_cftc_cot_registry_file_path(config_dir = NULL)get_cftc_cot_registry_file_path(config_dir = NULL)
config_dir |
Optional configuration directory used for fallback. |
Character scalar path.
Return only daily OHLC rows whose UTC date is strictly before an explicit cutoff. A finite row dated on the current UTC date is retained in the raw cache but is provisional until the next UTC date.
get_completed_local_quantmod_OHLC( label, src = "yahoo", interval = "1d", local_path = NULL, as_of = as.Date(Sys.time(), tz = "UTC") )get_completed_local_quantmod_OHLC( label, src = "yahoo", interval = "1d", local_path = NULL, as_of = as.Date(Sys.time(), tz = "UTC") )
label |
Local symbol label used in the stored data. |
src |
quantmod source, default |
interval |
Interval label. Only |
local_path |
Optional local storage path. |
as_of |
UTC timestamp used to determine the current UTC date. |
data.table or NULL.
Get Crypto Derivatives Registry
get_crypto_derivatives_registry( registry_path = get_crypto_derivatives_registry_file_path() )get_crypto_derivatives_registry( registry_path = get_crypto_derivatives_registry_file_path() )
registry_path |
Optional JSON registry path. |
A registry data.table.
Get Crypto Derivatives Registry File Path
get_crypto_derivatives_registry_file_path(config_dir = NULL)get_crypto_derivatives_registry_file_path(config_dir = NULL)
config_dir |
Optional configuration directory used for fallback. |
Character scalar path.
Get EIA Series Registry
get_eia_registry(registry_path = get_eia_registry_file_path())get_eia_registry(registry_path = get_eia_registry_file_path())
registry_path |
Optional JSON registry path. |
A registry data.table.
Get EIA Registry File Path
get_eia_registry_file_path(config_dir = NULL)get_eia_registry_file_path(config_dir = NULL)
config_dir |
Optional configuration directory used for fallback. |
Character scalar path.
Get Treasury Fiscal Data Registry
get_fiscaldata_registry(registry_path = get_fiscaldata_registry_file_path())get_fiscaldata_registry(registry_path = get_fiscaldata_registry_file_path())
registry_path |
Optional JSON registry path. |
A registry data.table.
Get Treasury Fiscal Data Registry File Path
get_fiscaldata_registry_file_path(config_dir = NULL)get_fiscaldata_registry_file_path(config_dir = NULL)
config_dir |
Optional configuration directory used for fallback. |
Character scalar path.
Get FRED Registry
get_fred_registry(registry_path = get_fred_registry_file_path())get_fred_registry(registry_path = get_fred_registry_file_path())
registry_path |
Optional JSON registry path. |
data.table.
Resolve the JSON registry path for FRED metadata. If no explicit
registry_file is configured, the function falls back to a default filename
in the package config directory.
get_fred_registry_file_path(config_dir = NULL)get_fred_registry_file_path(config_dir = NULL)
config_dir |
Optional configuration directory used for the fallback registry path. |
Character scalar path.
Returns factual, provider-neutral market-instrument metadata for downstream
consumers. It deliberately excludes editorial, ranking, simulation, and
execution fields. canonical_symbol is a consumer identity, not a provider
symbol. Current Yahoo local-cache access and synchronization resolve through
provider_identifiers$yahoo; do not pass canonical_symbol directly to
get_completed_local_quantmod_OHLC(). primary_source describes intended
routing; actual row-level provenance remains the source recorded by
synchronization.
get_instrument_catalog( catalog_path = .instrument_catalog_seed_path(), yahoo_registry_path = get_yahoofinance_seed_registry_path(), validate = TRUE, schema_version = "1.0.0" )get_instrument_catalog( catalog_path = .instrument_catalog_seed_path(), yahoo_registry_path = get_yahoofinance_seed_registry_path(), validate = TRUE, schema_version = "1.0.0" )
catalog_path |
Path to a catalog JSON file. Defaults to the packaged seed. |
yahoo_registry_path |
Path to the Yahoo Finance sync registry. Defaults to the packaged seed registry so the catalog is usable without a runtime configuration file. Pass a runtime registry path to validate that runtime registry explicitly. |
validate |
Logical. When |
schema_version |
Output schema. The default |
A data.table with one row per instrument. provider_identifiers,
primary_source, fallback_sources, and supported_intervals are list
columns that preserve their JSON mapping or array structure.
Returns the cached package configuration, loading it from YAML if needed.
get_investdatar_config( reload = FALSE, config_path = Sys.getenv("INVESTDATAR_CONFIG", unset = "") )get_investdatar_config( reload = FALSE, config_path = Sys.getenv("INVESTDATAR_CONFIG", unset = "") )
reload |
Logical. Reload from disk even if configuration is already cached. |
config_path |
Optional config path used when |
A configuration list.
Get The Latest Batch Sync Run Log
get_latest_sync_run(source_id, local_path)get_latest_sync_run(source_id, local_path)
source_id |
Source identifier used for the sync log filename prefix. |
local_path |
Local source data path that contains the |
A run-log list, or NULL when no matching log file exists.
Read Local Alpha Vantage Daily Data
get_local_alphavantage_data(symbol, local_path = NULL)get_local_alphavantage_data(symbol, local_path = NULL)
symbol |
Market symbol. |
local_path |
Optional Alpha Vantage storage directory. |
A standardized market data.table, or NULL.
Read Local BEA Data
get_local_bea_data(series_id, local_path = NULL)get_local_bea_data(series_id, local_path = NULL)
series_id |
Registered series identifier. |
local_path |
Optional BEA storage directory. |
A data.table, or NULL.
Get Local Binance Kline Data
get_local_binance_klines( symbol = "ETHUSDT", interval = "1m", local_path = NULL, storage = c("single", "monthly"), from = NULL, to = NULL )get_local_binance_klines( symbol = "ETHUSDT", interval = "1m", local_path = NULL, storage = c("single", "monthly"), from = NULL, to = NULL )
symbol |
Trading pair symbol. |
interval |
Candlestick interval. |
local_path |
Optional Binance storage path. |
storage |
Local storage mode: monolithic |
from, to
|
Optional bounds used to prune monthly partitions before read. |
data.table or NULL.
Read Local BLS Data
get_local_bls_data(series_id, local_path = NULL)get_local_bls_data(series_id, local_path = NULL)
series_id |
BLS series identifier. |
local_path |
Optional BLS storage directory. |
A data.table, or NULL.
Read Local Census Data
get_local_census_data(series_id, local_path = NULL)get_local_census_data(series_id, local_path = NULL)
series_id |
Registered series identifier. |
local_path |
Optional Census storage directory. |
A data.table, or NULL.
Read Local CFTC COT Data
get_local_cftc_cot(report_id, local_path = NULL)get_local_cftc_cot(report_id, local_path = NULL)
report_id |
Registry report identifier. |
local_path |
Optional CFTC storage directory. |
A data.table, or NULL when no local file exists.
Read Local Crypto Derivatives Data
get_local_crypto_derivatives( provider, dataset_type, symbol, interval = NULL, local_path = NULL, storage = c("single", "monthly"), from = NULL, to = NULL )get_local_crypto_derivatives( provider, dataset_type, symbol, interval = NULL, local_path = NULL, storage = c("single", "monthly"), from = NULL, to = NULL )
provider |
|
dataset_type |
Funding, open-interest, mark/index price, basis, or Binance long-short-ratio dataset identifier. |
symbol |
Provider instrument identifier. |
interval |
Required Binance open-interest period; otherwise a local label. |
local_path |
Optional derivatives storage directory. |
storage |
Local storage mode: monolithic |
from, to
|
Optional inclusive UTC time bounds. |
A data.table, or NULL.
Get Local Data Metadata
get_local_data_meta(local_file_path, fallback_to_file_mtime = TRUE)get_local_data_meta(local_file_path, fallback_to_file_mtime = TRUE)
local_file_path |
Path to a local |
fallback_to_file_mtime |
Logical. If no sidecar metadata file exists, use the data file modification time. |
A list of metadata, or NULL.
Reads a dataset stored as monthly RDS partitions. Time bounds are applied to partition discovery before files are loaded. If no partition directory exists, an existing monolithic RDS file is read for backward compatibility.
get_local_data_partitioned( local_file_path, time_col, from = NULL, to = NULL, order_cols = time_col, tz = "UTC" )get_local_data_partitioned( local_file_path, time_col, from = NULL, to = NULL, order_cols = time_col, tz = "UTC" )
local_file_path |
Virtual monolithic |
time_col |
Name of the partition timestamp column. |
from, to
|
Optional inclusive time bounds. |
order_cols |
Optional output ordering columns. |
tz |
Time zone used to determine month boundaries. |
A data.table, or NULL when no local data exists.
Get Local Update Time
get_local_data_utime(local_file_path)get_local_data_utime(local_file_path)
local_file_path |
Path to a local |
POSIXct or NULL.
Read Local EIA Data
get_local_eia_data(series_id, local_path = NULL)get_local_eia_data(series_id, local_path = NULL)
series_id |
EIA series identifier. |
local_path |
Optional EIA storage directory. |
A data.table, or NULL.
Read Local Treasury Fiscal Data
get_local_fiscaldata(dataset_id, local_path = NULL)get_local_fiscaldata(dataset_id, local_path = NULL)
dataset_id |
Registry dataset identifier. |
local_path |
Optional Fiscal Data storage directory. |
A data.table, or NULL.
Get Local FRED Data
get_local_FRED_data(series_id, local_path = NULL)get_local_FRED_data(series_id, local_path = NULL)
series_id |
FRED series identifier. |
local_path |
Optional local storage path. |
data.table or NULL.
Read Provider-Neutral Local Fund Holdings
get_local_fund_holdings(provider, fund_id, local_path = NULL)get_local_fund_holdings(provider, fund_id, local_path = NULL)
provider |
Fund issuer/provider identifier. |
fund_id |
Fund ticker or stable identifier. |
local_path |
Optional generic holdings storage root. |
A provider-neutral data.table, or NULL.
Get Local OKX Candle Data
get_local_okx_candle( inst_id, bar, local_path = NULL, storage = c("single", "monthly"), from = NULL, to = NULL )get_local_okx_candle( inst_id, bar, local_path = NULL, storage = c("single", "monthly"), from = NULL, to = NULL )
inst_id |
Instrument identifier. |
bar |
Candle interval. |
local_path |
Optional OKX storage path. |
storage |
Local storage mode: monolithic |
from, to
|
Optional bounds used to prune monthly partitions before read. |
data.table or NULL.
Get Local quantmod OHLC Data
get_local_quantmod_OHLC( label, src = "yahoo", interval = "1d", local_path = NULL )get_local_quantmod_OHLC( label, src = "yahoo", interval = "1d", local_path = NULL )
label |
Local symbol label used in the stored data. |
src |
quantmod source, default |
interval |
Interval label, default |
local_path |
Optional local storage path. |
data.table or NULL.
Get Local RSS Data
get_local_rss_data(feed_id, local_path = NULL)get_local_rss_data(feed_id, local_path = NULL)
feed_id |
Local feed identifier. |
local_path |
Optional local storage path. |
data.table or NULL.
Read Local SDMX Data
get_local_sdmx_data(series_id, local_path = NULL)get_local_sdmx_data(series_id, local_path = NULL)
series_id |
Registry series identifier. |
local_path |
Optional SDMX storage directory. |
A data.table, or NULL.
Read Local SEC Company Facts
get_local_sec_companyfacts(cik, local_path = NULL)get_local_sec_companyfacts(cik, local_path = NULL)
cik |
SEC Central Index Key. |
local_path |
Optional SEC Company Facts storage directory. |
A data.table, or NULL.
Read A Local SEC XBRL Frame
get_local_sec_frame(taxonomy, tag, unit, period, local_path = NULL)get_local_sec_frame(taxonomy, tag, unit, period, local_path = NULL)
taxonomy |
XBRL taxonomy, such as |
tag |
XBRL concept tag. |
unit |
XBRL unit, such as |
period |
SEC frame period, such as |
local_path |
Optional SEC frame cache directory. |
A cached data.table, or NULL.
Read Local SEC Submissions
get_local_sec_submissions(cik, local_path = NULL)get_local_sec_submissions(cik, local_path = NULL)
cik |
SEC Central Index Key. |
local_path |
Optional SEC submissions storage directory. |
A data.table, or NULL.
Get Local Treasury Rates
get_local_treasury_rates(dataset, local_path = NULL)get_local_treasury_rates(dataset, local_path = NULL)
dataset |
Treasury dataset key. |
local_path |
Optional local storage path. |
data.table or NULL.
Get Local World Bank Data
get_local_wbstats_data(indicator, country, freq = "Y", local_path = NULL)get_local_wbstats_data(indicator, country, freq = "Y", local_path = NULL)
indicator |
World Bank indicator code. |
country |
Country or aggregate code. |
freq |
Frequency code. |
local_path |
Optional local storage path. |
data.table or NULL.
Returns the provider-neutral 4-hour OKX perpetual-swap subset. Its
canonical_symbol values are consumer identities; local cache access uses
provider_identifiers$okx with get_local_okx_candle(). Contract metadata
describes OKX linear USDT-margined perpetuals; actual cached row provenance
remains in the synchronized data.
get_okx_perpetual_catalog( catalog_path = .instrument_catalog_seed_path(), local_path = NULL, as_of = Sys.time(), validate = TRUE )get_okx_perpetual_catalog( catalog_path = .instrument_catalog_seed_path(), local_path = NULL, as_of = Sys.time(), validate = TRUE )
catalog_path |
Path to a catalog JSON file. Defaults to the packaged seed. |
local_path |
Optional local OKX candle-cache directory. Supplying it enables catalog-to-cache validation. |
as_of |
UTC time used to exclude cache rows at or after the cutoff. |
validate |
Logical. Validate the catalog before returning it. |
A data.table with one row per 4-hour OKX perpetual instrument.
Get RSS Registry
get_rss_registry(registry_path = get_rss_registry_file_path())get_rss_registry(registry_path = get_rss_registry_file_path())
registry_path |
Optional JSON registry path. |
data.table.
Resolve the JSON registry path for RSS feed metadata. If no explicit
registry_file is configured, the function falls back to a default filename
in the package config directory.
get_rss_registry_file_path(config_dir = NULL)get_rss_registry_file_path(config_dir = NULL)
config_dir |
Optional configuration directory used for the fallback registry path. |
Character scalar path.
Get SDMX Series Registry
get_sdmx_registry(registry_path = get_sdmx_registry_file_path())get_sdmx_registry(registry_path = get_sdmx_registry_file_path())
registry_path |
Optional JSON registry path. |
A registry data.table.
Get SDMX Registry File Path
get_sdmx_registry_file_path(config_dir = NULL)get_sdmx_registry_file_path(config_dir = NULL)
config_dir |
Optional configuration directory used for fallback. |
Character scalar path.
Retrieve SEC Ticker-CIK Mappings
get_sec_company_tickers(config = NULL)get_sec_company_tickers(config = NULL)
config |
Optional SEC configuration containing an identifiable user agent. |
A data.table with cik, company_name, ticker, and exchange.
Get SEC Frames Registry
get_sec_frames_registry(registry_path = get_sec_frames_registry_file_path())get_sec_frames_registry(registry_path = get_sec_frames_registry_file_path())
registry_path |
Optional registry path. |
A registry data.table.
Get SEC Frames Registry File Path
get_sec_frames_registry_file_path(config_dir = NULL)get_sec_frames_registry_file_path(config_dir = NULL)
config_dir |
Optional configuration directory. |
Character scalar path.
Get SEC Company Registry
get_sec_registry(registry_path = get_sec_registry_file_path())get_sec_registry(registry_path = get_sec_registry_file_path())
registry_path |
Optional JSON registry path. |
A registry data.table.
Get SEC Company Registry File Path
get_sec_registry_file_path(config_dir = NULL)get_sec_registry_file_path(config_dir = NULL)
config_dir |
Optional configuration directory used for fallback. |
Character scalar path.
Returns the configuration block for a named source.
get_source_config(source, config = get_investdatar_config())get_source_config(source, config = get_investdatar_config())
source |
Character source name. |
config |
Optional package config list. |
A list, or an empty list if the source is not configured.
Retrieves daily historical stock data (open, high, low, close, volume) for a given symbol
using the AlphaVantage API. You can specify whether to retrieve only recent data
("compact") or the full history ("full").
The returned object follows the package's standardized market OHLCV schema.
get_source_data_alphavantage_ts_daily( symbol, mode = c("compact", "full"), config = NULL )get_source_data_alphavantage_ts_daily( symbol, mode = c("compact", "full"), config = NULL )
symbol |
A character string for the stock ticker symbol (e.g., |
mode |
Character string; either |
config |
A list of AlphaVantage API settings, typically retrieved via |
A data.table with standardized columns including source,
symbol, interval, datetime, date, open, high, low, close,
and volume.
datetimeMidnight timestamp derived from the trading date.
dateDate of the observation.
openOpening price.
highHighest price of the day.
lowLowest price of the day.
closeClosing price.
volumeVolume of trades.
## Not run: config <- list(api_key = Sys.getenv("ALPHAVANTAGE_API_KEY")) df <- get_source_data_alphavantage_ts_daily("MSFT", mode = "compact", config = config) head(df) ## End(Not run)## Not run: config <- list(api_key = Sys.getenv("ALPHAVANTAGE_API_KEY")) df <- get_source_data_alphavantage_ts_daily("MSFT", mode = "compact", config = config) head(df) ## End(Not run)
Retrieve BEA Regional Data
get_source_data_bea( series_id, table_name, line_code, geofips = "STATE", year = "ALL", label = NULL, config = NULL )get_source_data_bea( series_id, table_name, line_code, geofips = "STATE", year = "ALL", label = NULL, config = NULL )
series_id |
Stable local series identifier. |
table_name |
BEA Regional table name. |
line_code |
BEA line code. |
geofips |
Geography selector such as |
year |
Year selector, default |
label |
Optional local label. |
config |
Optional BEA configuration. |
A standardized regional panel.
Retrieves candlestick data from Binance USDT-margined futures. If
paginate = TRUE, the function keeps requesting pages until the requested
time range is exhausted or the API returns fewer rows than limit.
get_source_data_binance_klines( symbol = "ETHUSDT", interval = "1m", start_time = NULL, end_time = NULL, limit = 1500L, tz = "UTC", paginate = TRUE )get_source_data_binance_klines( symbol = "ETHUSDT", interval = "1m", start_time = NULL, end_time = NULL, limit = 1500L, tz = "UTC", paginate = TRUE )
symbol |
Trading pair symbol. |
interval |
Candlestick interval. |
start_time |
Optional start time. |
end_time |
Optional end time. |
limit |
Page size. Binance futures allows up to 1500. |
tz |
Time zone applied to returned timestamps. |
paginate |
Logical. Request multiple pages when needed. |
Standardized OHLCV data.table with source, symbol, interval,
datetime, date, open, high, low, close, and volume, plus
Binance-specific columns such as close_time and trade counts.
Retrieve One BLS Series
get_source_data_bls( series_id, label = NULL, from = NULL, to = NULL, config = NULL )get_source_data_bls( series_id, label = NULL, from = NULL, to = NULL, config = NULL )
series_id |
BLS series identifier. |
label |
Optional local label. |
from, to
|
Optional year or date bounds. |
config |
Optional BLS configuration. |
A standardized monthly data.table.
Retrieve A Selected Census Economic Indicator Series
get_source_data_census( series_id, dataset, data_type_code, category_code, seasonally_adj = "yes", from = NULL, to = NULL, label = NULL, config = NULL )get_source_data_census( series_id, dataset, data_type_code, category_code, seasonally_adj = "yes", from = NULL, to = NULL, label = NULL, config = NULL )
series_id |
Stable local series identifier. |
dataset |
Economic Indicators dataset suffix, such as |
data_type_code |
Census item type code. |
category_code |
Census industry/category code. |
seasonally_adj |
Seasonal adjustment value. |
from, to
|
Optional time bounds. |
label |
Optional local label. |
config |
Optional Census configuration. |
A standardized long data.table.
Retrieves TFF, Disaggregated, or Legacy reports from the CFTC Public Reporting Environment.
get_source_data_cftc_cot( report_variant = c("futures_only", "combined"), report_type = "tff", report_id = NULL, dataset_id = NULL, market_codes = NULL, from = NULL, to = NULL, page_size = 5000L, max_pages = Inf )get_source_data_cftc_cot( report_variant = c("futures_only", "combined"), report_type = "tff", report_id = NULL, dataset_id = NULL, market_codes = NULL, from = NULL, to = NULL, page_size = 5000L, max_pages = Inf )
report_variant |
Report variant: |
report_type |
Report family: |
report_id |
Stable local report identifier. |
dataset_id |
Optional official Socrata dataset identifier. |
market_codes |
Optional CFTC contract-market codes. |
from, to
|
Optional inclusive report-date bounds. |
page_size |
Number of source rows requested per page. |
max_pages |
Optional page limit, primarily useful for diagnostics. |
A standardized wide data.table, one row per market and report date.
Retrieve Historical Crypto Derivatives Data
get_source_data_crypto_derivatives( provider, dataset_type, symbol, interval = NULL, from = NULL, to = NULL, limit = NULL, config = NULL )get_source_data_crypto_derivatives( provider, dataset_type, symbol, interval = NULL, from = NULL, to = NULL, limit = NULL, config = NULL )
provider |
|
dataset_type |
Funding, open-interest, mark/index price, basis, or Binance long-short-ratio dataset identifier. |
symbol |
Provider instrument identifier. |
interval |
Required Binance open-interest period; otherwise a local label. |
from, to
|
Optional inclusive UTC time bounds. |
limit |
Provider page size. |
config |
Optional exchange configuration. |
A standardized data.table retaining provider fields.
Retrieve One EIA Series
get_source_data_eia( series_id, label = NULL, config = NULL, from = NULL, to = NULL, page_size = 5000L, max_pages = Inf )get_source_data_eia( series_id, label = NULL, config = NULL, from = NULL, to = NULL, page_size = 5000L, max_pages = Inf )
series_id |
EIA series identifier. |
label |
Optional local label. |
config |
Optional EIA API configuration. |
from, to
|
Optional inclusive period bounds. |
page_size |
Number of observations requested per page. |
max_pages |
Optional page limit for diagnostics. |
A standardized long data.table.
Retrieve A U.S. Treasury Fiscal Data Table
get_source_data_fiscaldata( dataset_id, endpoint, date_col = "record_date", key_cols = date_col, fields = NULL, from = NULL, to = NULL, page_size = 10000L, max_pages = Inf )get_source_data_fiscaldata( dataset_id, endpoint, date_col = "record_date", key_cols = date_col, fields = NULL, from = NULL, to = NULL, page_size = 10000L, max_pages = Inf )
dataset_id |
Stable local dataset identifier. |
endpoint |
Fiscal Data endpoint relative to the fiscal-service base URL. |
date_col |
Source date column. |
key_cols |
Source columns that uniquely identify observations per date. |
fields |
Optional source fields to request. |
from, to
|
Optional inclusive date bounds. |
page_size |
Number of source rows requested per page. |
max_pages |
Optional page limit for diagnostics. |
A typed, table-shaped data.table with source and dataset identity.
Download a FRED series and return dates and numeric values.
get_source_data_fred(series_id, config = NULL)get_source_data_fred(series_id, config = NULL)
series_id |
Character. FRED series ID (e.g., "DGS10"). |
config |
Optional list with fields: |
data.table with columns: date (Date), value (numeric, NA for ".").
Get Latest OKX Candle Data
get_source_data_okx_candle( inst_id, bar, limit = 100L, config = NULL, tz = "UTC" )get_source_data_okx_candle( inst_id, bar, limit = 100L, config = NULL, tz = "UTC" )
inst_id |
Instrument identifier. |
bar |
Candle interval. |
limit |
Integer page size. |
config |
Optional OKX API config. If omitted, defaults from the
package config and |
tz |
Output time zone. |
data.table or NULL.
Resolves the local data directory for a source using the package config.
get_source_data_path( source, config = get_investdatar_config(), subdir = NULL, create = FALSE )get_source_data_path( source, config = get_investdatar_config(), subdir = NULL, create = FALSE )
source |
Character source name. |
config |
Optional package config list. |
subdir |
Optional subdirectory appended to the source data path. |
create |
Logical. Create the directory if it does not exist. |
Character scalar path.
Download an RSS feed and return a normalized item-level data.table.
get_source_data_rss( feed_id, url, parser = c("plain", "gdpnow"), ca_bundle = NULL )get_source_data_rss( feed_id, url, parser = c("plain", "gdpnow"), ca_bundle = NULL )
feed_id |
Local feed identifier. |
url |
RSS feed URL. |
parser |
Parsing strategy. Currently supports |
ca_bundle |
Optional CA bundle path used only for this feed request. |
data.table.
Retrieve Data From An SDMX Provider
get_source_data_sdmx( series_id, provider, base_url, agency = NULL, dataflow, version = "latest", key = "", flow_ref = NULL, format = NULL, accept = "text/csv", time_col = "TIME_PERIOD", value_col = "OBS_VALUE", dimension_cols = NULL, label = NULL, frequency = NULL, from = NULL, to = NULL, last_n_observations = NULL )get_source_data_sdmx( series_id, provider, base_url, agency = NULL, dataflow, version = "latest", key = "", flow_ref = NULL, format = NULL, accept = "text/csv", time_col = "TIME_PERIOD", value_col = "OBS_VALUE", dimension_cols = NULL, label = NULL, frequency = NULL, from = NULL, to = NULL, last_n_observations = NULL )
series_id |
Stable local series identifier. |
provider |
Provider dialect: |
base_url |
Provider REST base URL. |
agency |
Optional SDMX agency identifier. |
dataflow |
SDMX dataflow identifier. |
version |
SDMX dataflow version. |
key |
SDMX series key. |
flow_ref |
Optional provider-specific flow reference. |
format |
Provider query format. |
accept |
HTTP response media type. |
time_col, value_col
|
Source observation columns. |
dimension_cols |
Columns identifying distinct series in the response. |
label, frequency
|
Optional local metadata. |
from, to
|
Optional inclusive SDMX periods. |
last_n_observations |
Optional number of latest observations requested per matching SDMX series. |
A standardized long data.table retaining original SDMX columns.
Retrieve SEC Company Facts
get_source_data_sec_companyfacts( cik, ticker = NULL, company_name = NULL, concepts = NULL, forms = NULL, from = NULL, to = NULL, config = NULL )get_source_data_sec_companyfacts( cik, ticker = NULL, company_name = NULL, concepts = NULL, forms = NULL, from = NULL, to = NULL, config = NULL )
cik |
SEC Central Index Key. |
ticker |
Optional ticker label. |
company_name |
Optional company name label. |
concepts |
Optional concept names or |
forms |
Optional filing forms to retain. |
from, to
|
Optional inclusive filed-date bounds. |
config |
Optional SEC configuration. |
A standardized long XBRL fact data.table.
Retrieve An SEC XBRL Frame
get_source_data_sec_frame(taxonomy, tag, unit, period, config = NULL)get_source_data_sec_frame(taxonomy, tag, unit, period, config = NULL)
taxonomy |
XBRL taxonomy, such as |
tag |
XBRL concept tag. |
unit |
XBRL unit, such as |
period |
SEC frame period, such as |
config |
Optional SEC configuration. |
A normalized cross-company data.table.
Retrieve SEC Filing Submissions
get_source_data_sec_submissions( cik, ticker = NULL, company_name = NULL, forms = NULL, from = NULL, to = NULL, include_history = TRUE, config = NULL )get_source_data_sec_submissions( cik, ticker = NULL, company_name = NULL, forms = NULL, from = NULL, to = NULL, include_history = TRUE, config = NULL )
cik |
SEC Central Index Key. |
ticker |
Optional ticker label. |
company_name |
Optional company name label. |
forms |
Optional filing forms to retain. |
from, to
|
Optional inclusive filing-date bounds. |
include_history |
Logical; retrieve older SEC submission files in addition to recent filings. |
config |
Optional SEC configuration. |
A standardized filing-event data.table.
Download U.S. Treasury raw daily rates and return a normalized long-format
data.table.
get_source_data_treasury_rates(dataset, years = NULL)get_source_data_treasury_rates(dataset, years = NULL)
dataset |
Treasury dataset key. Supported values are |
years |
Optional integer vector of calendar years to fetch. If omitted, the function combines the historical archive with current-year annual XML feeds from 2024 through the current year. |
data.table.
Downloads indicator data using the wbstats package and standardizes it to
a package-native long schema.
get_source_data_wbstats( indicator, country = "countries_only", start_date = NULL, end_date = NULL, mrv = NULL, mrnev = NULL, cache = NULL, freq = "Y", gapfill = FALSE, date_as_class_date = FALSE, lang = NULL )get_source_data_wbstats( indicator, country = "countries_only", start_date = NULL, end_date = NULL, mrv = NULL, mrnev = NULL, cache = NULL, freq = "Y", gapfill = FALSE, date_as_class_date = FALSE, lang = NULL )
indicator |
World Bank indicator code. |
country |
Country or aggregate code. |
start_date |
Optional start date. |
end_date |
Optional end date. |
mrv |
Optional number of most recent values. |
mrnev |
Optional number of most recent non-empty values. |
cache |
Optional cache object passed to |
freq |
Frequency code: |
gapfill |
Logical passed to |
date_as_class_date |
Logical passed to |
lang |
Optional language code. |
Standardized data.table.
Get Historical OKX Candle Data
get_source_hist_data_okx_candle( inst_id, bar, before = NULL, limit = 100L, config = NULL, tz = "UTC" )get_source_hist_data_okx_candle( inst_id, bar, before = NULL, limit = 100L, config = NULL, tz = "UTC" )
inst_id |
Instrument identifier. |
bar |
Candle interval. |
before |
Optional pagination cursor. Date-time cursors are interpreted
in |
limit |
Integer page size. |
config |
Optional OKX API config. If omitted, defaults from the
package config and |
tz |
Output time zone. |
data.table or NULL.
Queries the FRED API to obtain basic metadata of a given series, such as title, observation range, frequency, units, and seasonal adjustment.
get_source_metadata_fred(series_id, config = NULL)get_source_metadata_fred(series_id, config = NULL)
series_id |
A character string. The FRED series ID (e.g., "AMERIBOR"). |
config |
A list containing API configuration with elements:
|
A list with elements:
title — Series title
start — Observation start date
end — Observation end date
freq — Data frequency
units — Measurement units
season — Seasonal adjustment type
## Not run: config <- list( api_key = "your_api_key", url = "https://api.stlouisfed.org/fred/series", mode = "json" ) get_source_metadata_fred("AMERIBOR", config) ## End(Not run)## Not run: config <- list( api_key = "your_api_key", url = "https://api.stlouisfed.org/fred/series", mode = "json" ) get_source_metadata_fred("AMERIBOR", config) ## End(Not run)
Returns a formal source-spec object describing one provider module. The spec is intentionally lightweight so modules can later move into separate packages without changing the high-level contract.
get_source_spec(source)get_source_spec(source)
source |
Character provider key. |
An object of class investdatar_source_spec.
Get CFTC Dataset Update Time
get_source_utime_cftc_cot( report_variant = c("futures_only", "combined"), report_type = "tff", dataset_id = NULL )get_source_utime_cftc_cot( report_variant = c("futures_only", "combined"), report_type = "tff", dataset_id = NULL )
report_variant |
Report variant: |
report_type |
Report family: |
dataset_id |
Optional official Socrata dataset identifier. |
A UTC POSIXct update time, or NULL when unavailable.
Get Latest Crypto Derivatives Observation Time
get_source_utime_crypto_derivatives( provider, dataset_type, symbol, interval = NULL, config = NULL )get_source_utime_crypto_derivatives( provider, dataset_type, symbol, interval = NULL, config = NULL )
provider |
|
dataset_type |
Funding, open-interest, mark/index price, basis, or Binance long-short-ratio dataset identifier. |
symbol |
Provider instrument identifier. |
interval |
Required Binance open-interest period; otherwise a local label. |
config |
Optional exchange configuration. |
UTC POSIXct, or NULL.
Get Latest EIA Series Update Date
get_source_utime_eia(series_id, config = NULL)get_source_utime_eia(series_id, config = NULL)
series_id |
EIA series identifier. |
config |
Optional EIA API configuration. |
A UTC POSIXct, or NULL.
Get The Latest Treasury Fiscal Data Date
get_source_utime_fiscaldata(dataset_id, endpoint, date_col = "record_date")get_source_utime_fiscaldata(dataset_id, endpoint, date_col = "record_date")
dataset_id |
Stable local dataset identifier. |
endpoint |
Fiscal Data endpoint relative to the fiscal-service base URL. |
date_col |
Source date column. |
A UTC POSIXct inferred from the newest source record, or NULL.
Return the last update time of a FRED series as POSIXct in UTC.
get_source_utime_fred( series_id, config = NULL, from_server = FALSE, tz = "America/Chicago" )get_source_utime_fred( series_id, config = NULL, from_server = FALSE, tz = "America/Chicago" )
series_id |
Character. FRED series ID. |
config |
Optional list with fields: |
from_server |
Logical. If |
tz |
Time zone used when parsing or inferring the update time. |
POSIXct (UTC).
Floor current time to the last completed candle start for the given bar.
get_source_utime_okx_candle(bar, tz = "UTC")get_source_utime_okx_candle(bar, tz = "UTC")
bar |
Character. OKX timeframe tag (e.g., |
tz |
Character. IANA timezone. |
POSIXct.
Get RSS Feed Last Update Time
get_source_utime_rss( feed_id, url, parser = c("plain", "gdpnow"), ca_bundle = NULL )get_source_utime_rss( feed_id, url, parser = c("plain", "gdpnow"), ca_bundle = NULL )
feed_id |
Local feed identifier. |
url |
RSS feed URL. |
parser |
Parsing strategy. Currently supports |
ca_bundle |
Optional CA bundle path used only for this feed request. |
POSIXct or NULL.
Get Latest SDMX Observation Time
get_source_utime_sdmx( series_id, provider, base_url, agency = NULL, dataflow, version = "latest", key = "", flow_ref = NULL, format = NULL, accept = "text/csv", time_col = "TIME_PERIOD", value_col = "OBS_VALUE", dimension_cols = NULL, label = NULL, frequency = NULL )get_source_utime_sdmx( series_id, provider, base_url, agency = NULL, dataflow, version = "latest", key = "", flow_ref = NULL, format = NULL, accept = "text/csv", time_col = "TIME_PERIOD", value_col = "OBS_VALUE", dimension_cols = NULL, label = NULL, frequency = NULL )
series_id |
Stable local series identifier. |
provider |
Provider dialect: |
base_url |
Provider REST base URL. |
agency |
Optional SDMX agency identifier. |
dataflow |
SDMX dataflow identifier. |
version |
SDMX dataflow version. |
key |
SDMX series key. |
flow_ref |
Optional provider-specific flow reference. |
format |
Provider query format. |
accept |
HTTP response media type. |
time_col, value_col
|
Source observation columns. |
dimension_cols |
Columns identifying distinct series in the response. |
label, frequency
|
Optional local metadata. |
A UTC POSIXct, or NULL.
Get Latest SEC Company Facts Filing Date
get_source_utime_sec_companyfacts(cik, config = NULL)get_source_utime_sec_companyfacts(cik, config = NULL)
cik |
SEC Central Index Key. |
config |
Optional SEC configuration. |
A UTC POSIXct, or NULL.
Get Latest SEC Submission Time
get_source_utime_sec_submissions(cik, config = NULL)get_source_utime_sec_submissions(cik, config = NULL)
cik |
SEC Central Index Key. |
config |
Optional SEC configuration. |
A UTC-compatible POSIXct, or NULL.
Get Treasury Rates Last Update Time
get_source_utime_treasury_rates(dataset, years = NULL)get_source_utime_treasury_rates(dataset, years = NULL)
dataset |
Treasury dataset key. Supported values are |
years |
Optional integer vector of calendar years to fetch. If omitted, the function combines the historical archive with current-year annual XML feeds from 2024 through the current year. |
POSIXct or NULL.
Get Fallback Source Update Time for wbstats Data
get_source_utime_wbstats(freq = "Y", tz = "UTC")get_source_utime_wbstats(freq = "Y", tz = "UTC")
freq |
Frequency code: |
tz |
Time zone used for the inferred timestamp. |
POSIXct.
Get World Bank Registry
get_wbstats_registry(registry_path = get_wbstats_registry_file_path())get_wbstats_registry(registry_path = get_wbstats_registry_file_path())
registry_path |
Optional JSON registry path. |
data.table.
Resolve the JSON registry path for World Bank indicator metadata. If no
explicit registry_file is configured, the function falls back to a default
filename in the package config directory.
get_wbstats_registry_file_path(config_dir = NULL)get_wbstats_registry_file_path(config_dir = NULL)
config_dir |
Optional configuration directory used for the fallback registry path. |
Character scalar path.
Get Yahoo Finance Registry
get_yahoofinance_registry( registry_path = get_yahoofinance_registry_file_path() )get_yahoofinance_registry( registry_path = get_yahoofinance_registry_file_path() )
registry_path |
Optional registry JSON path. |
data.table.
Resolve the JSON registry path for Yahoo Finance ticker metadata. If no
explicit registry_file is configured, the function falls back to a default
filename in the package config directory.
get_yahoofinance_registry_file_path(config_dir = NULL)get_yahoofinance_registry_file_path(config_dir = NULL)
config_dir |
Optional configuration directory used for the fallback registry path. |
Character scalar path.
Show the seed-owned policy fields that would be added to an older external
Yahoo Finance registry. The runtime registry is never modified. If
output_path is supplied, the patch is written as JSON at that path.
get_yahoofinance_registry_migration_patch( registry_path = get_yahoofinance_registry_file_path(), seed_path = get_yahoofinance_seed_registry_path(), output_path = NULL )get_yahoofinance_registry_migration_patch( registry_path = get_yahoofinance_registry_file_path(), seed_path = get_yahoofinance_seed_registry_path(), output_path = NULL )
registry_path |
Runtime registry JSON path. |
seed_path |
Package seed registry JSON path. |
output_path |
Optional JSON path for the explicit migration patch. |
A data.table with ticker and policy fields.
Get Yahoo Finance Seed Registry Path
get_yahoofinance_seed_registry_path()get_yahoofinance_seed_registry_path()
Path to the package-managed Yahoo Finance registry seed.
Provides a fallback source update time for resources that do not expose a server-side update timestamp.
infer_source_utime_from_frequency( frequency, reference_time = Sys.time(), tz = "UTC" )infer_source_utime_from_frequency( frequency, reference_time = Sys.time(), tz = "UTC" )
frequency |
Frequency string or candle interval. |
reference_time |
POSIXct reference time. Defaults to |
tz |
Time zone used for period flooring. |
POSIXct.
A run is successful when its stored summary contains no row with an error status or non-empty error message. Empty summaries are successful because an intentionally empty registry has no failed work.
is_sync_run_successful(run)is_sync_run_successful(run)
run |
A run-log object returned by |
Logical scalar.
List Source Specifications
list_source_specs()list_source_specs()
Named list of investdatar_source_spec objects.
Loads the package YAML configuration and caches it in package options.
load_investdatar_config( config_path = Sys.getenv("INVESTDATAR_CONFIG", unset = "") )load_investdatar_config( config_path = Sys.getenv("INVESTDATAR_CONFIG", unset = "") )
config_path |
Optional path to the YAML config file. If omitted, the
function uses the |
A normalized configuration list.
Converts legacy schema 1.0.0 records without changing provider identifiers, cache keys, synchronization routing, or fallback declarations. Legacy MIC-like calendar values are treated as completed-bar calendar aliases only.
normalize_instrument_catalog( catalog_path = .instrument_catalog_seed_path(), yahoo_registry_path = get_yahoofinance_seed_registry_path() )normalize_instrument_catalog( catalog_path = .instrument_catalog_seed_path(), yahoo_registry_path = get_yahoofinance_seed_registry_path() )
catalog_path |
Path to a catalog JSON file. Defaults to the packaged seed. |
yahoo_registry_path |
Path to the Yahoo Finance sync registry. Defaults to the packaged seed registry so the catalog is usable without a runtime configuration file. Pass a runtime registry path to validate that runtime registry explicitly. |
A normalized data.table using schema 1.1.0 fields.
Apply a manually approved repair derived from a versioned
audit_csi300_cache_integrity() artifact. The function authorizes only
dates present in that artifact's discrepancy set, verifies that the current
cache has not changed since the audit, and verifies the fresh Eastmoney
candidate against the audited candidate using the audit's close tolerance.
repair_csi300_cache_integrity( audit_artifact_path, approved_dates, local_path = NULL, output_dir = NULL, dry_run = TRUE )repair_csi300_cache_integrity( audit_artifact_path, approved_dates, local_path = NULL, output_dir = NULL, dry_run = TRUE )
audit_artifact_path |
JSON report produced by
|
approved_dates |
Explicit unique dates to repair. Every date must be in the audit artifact's discrepancy set. |
local_path |
Yahoo Finance cache directory. Defaults to the configured Yahoo Finance data path. |
output_dir |
Directory for repair logs and backups. Defaults to
|
dry_run |
Whether to write only a proposed repair log. Defaults to
|
The default is a dry run. It always writes a versioned repair log; dry runs do not mutate the cache or create a backup. A non-dry-run first copies the cache to the repair directory, then replaces only the explicitly approved rows. If writing fails, it restores the cache from that backup.
A list with the proposed or applied repair rows, log path, and (when applied) backup path.
Fetches all requested Binance kline windows in memory and writes the merged
repair result to local storage with one sync_local_data() call.
repair_local_binance_klines_gaps( symbol = "ETHUSDT", interval = "1m", windows, limit = 1500L, tz = "UTC", local_path = NULL, storage = c("single", "monthly") )repair_local_binance_klines_gaps( symbol = "ETHUSDT", interval = "1m", windows, limit = 1500L, tz = "UTC", local_path = NULL, storage = c("single", "monthly") )
symbol |
Trading pair symbol. |
interval |
Candlestick interval. |
windows |
A |
limit |
Page size. Binance futures allows up to 1500. |
tz |
Time zone applied to returned timestamps. |
local_path |
Optional Binance storage path. |
storage |
Local storage mode: monolithic |
A sync result list.
Fetches multiple OKX historical candle pages in memory and writes the merged
repair result to local storage with one sync_local_data() call.
repair_local_okx_candle_gaps( inst_id, bar, before, config = NULL, local_path = NULL, limit = 100L, tz = "UTC", storage = c("single", "monthly") )repair_local_okx_candle_gaps( inst_id, bar, before, config = NULL, local_path = NULL, limit = 100L, tz = "UTC", storage = c("single", "monthly") )
inst_id |
Instrument identifier. |
bar |
Candle interval. |
before |
Character, numeric, or date-time vector of OKX history
pagination cursors. Date-time cursors are interpreted in |
config |
Optional OKX API config. If omitted, defaults from the
package config and |
local_path |
Optional OKX storage path. |
limit |
Integer page size. |
tz |
Output time zone. |
storage |
Local storage mode: monolithic |
A sync result list.
Resolve An SEC CIK From A Ticker
resolve_sec_cik(ticker, mappings = NULL, config = NULL)resolve_sec_cik(ticker, mappings = NULL, config = NULL)
ticker |
Security ticker. |
mappings |
Optional table returned by |
config |
Optional SEC configuration used when mappings are not supplied. |
Unpadded CIK character scalar.
Converts issuer-specific snapshots to the provider-neutral holdings contract.
standardize_fund_holdings( data, provider, fund_id, as_of_date = NULL, column_map = list() )standardize_fund_holdings( data, provider, fund_id, as_of_date = NULL, column_map = list() )
data |
Holdings table. |
provider |
Stable provider identifier, such as |
fund_id |
Stable fund ticker or identifier. |
as_of_date |
Optional snapshot date overriding the source column. |
column_map |
Optional named list mapping canonical names to source columns. |
A provider-neutral long data.table.
Synchronize Registered Alpha Vantage Series
sync_all_alphavantage_registry_data( registry = get_alphavantage_registry(), config = NULL, local_path = NULL, ... )sync_all_alphavantage_registry_data( registry = get_alphavantage_registry(), config = NULL, local_path = NULL, ... )
registry |
Optional Alpha Vantage registry. |
config |
Optional API configuration. |
local_path |
Optional storage directory. |
... |
Passed to |
A standardized batch summary data.table.
Synchronize Registered BEA Series
sync_all_bea_registry_data( registry = get_bea_registry(), config = NULL, local_path = NULL )sync_all_bea_registry_data( registry = get_bea_registry(), config = NULL, local_path = NULL )
registry |
Optional BEA registry. |
config |
Optional BEA configuration. |
local_path |
Optional storage directory. |
A batch summary.
Synchronize Registered BLS Series
sync_all_bls_registry_data( registry = get_bls_registry(), config = NULL, local_path = NULL, ... )sync_all_bls_registry_data( registry = get_bls_registry(), config = NULL, local_path = NULL, ... )
registry |
Optional BLS registry. |
config |
Optional BLS configuration. |
local_path |
Optional BLS storage directory. |
... |
Passed to |
A batch summary.
Synchronize Registered Census Series
sync_all_census_registry_data( registry = get_census_registry(), config = NULL, local_path = NULL )sync_all_census_registry_data( registry = get_census_registry(), config = NULL, local_path = NULL )
registry |
Optional Census registry. |
config |
Optional Census configuration. |
local_path |
Optional storage directory. |
A batch summary.
Synchronize All Registered CFTC COT Reports
sync_all_cftc_cot_registry_data( registry = get_cftc_cot_registry(), local_path = NULL, ... )sync_all_cftc_cot_registry_data( registry = get_cftc_cot_registry(), local_path = NULL, ... )
registry |
Optional CFTC registry table. |
local_path |
Optional CFTC storage directory. |
... |
Passed to |
A standardized batch summary data.table.
Synchronize All Registered Crypto Derivatives Data
sync_all_crypto_derivatives_registry_data( registry = get_crypto_derivatives_registry(), local_path = NULL, ... )sync_all_crypto_derivatives_registry_data( registry = get_crypto_derivatives_registry(), local_path = NULL, ... )
registry |
Optional derivatives registry. |
local_path |
Optional derivatives storage directory. |
... |
Passed to |
A standardized batch summary data.table.
Synchronize All Registered EIA Series
sync_all_eia_registry_data( registry = get_eia_registry(), config = NULL, local_path = NULL, ... )sync_all_eia_registry_data( registry = get_eia_registry(), config = NULL, local_path = NULL, ... )
registry |
Optional EIA registry table. |
config |
Optional EIA API configuration. |
local_path |
Optional EIA storage directory. |
... |
Passed to |
A standardized batch summary data.table.
Synchronize All Registered Treasury Fiscal Data
sync_all_fiscaldata_registry_data( registry = get_fiscaldata_registry(), local_path = NULL, ... )sync_all_fiscaldata_registry_data( registry = get_fiscaldata_registry(), local_path = NULL, ... )
registry |
Optional Fiscal Data registry table. |
local_path |
Optional Fiscal Data storage directory. |
... |
Passed to |
A standardized batch summary data.table.
Synchronize All FRED Series In The Registry
sync_all_fred_registry_data( registry = get_fred_registry(), config = NULL, local_path = NULL, from_server = FALSE, tz = "America/Chicago" )sync_all_fred_registry_data( registry = get_fred_registry(), config = NULL, local_path = NULL, from_server = FALSE, tz = "America/Chicago" )
registry |
Optional FRED registry table. |
config |
Optional FRED API config. |
local_path |
Optional local storage path. |
from_server |
Logical. If |
tz |
Time zone used for source update time inference. |
Summary data.table.
Synchronize All RSS Registry Feeds
sync_all_rss_registry_data( registry = get_rss_registry(), local_path = NULL, ca_bundles = NULL )sync_all_rss_registry_data( registry = get_rss_registry(), local_path = NULL, ca_bundles = NULL )
registry |
Optional RSS registry table. |
local_path |
Optional local storage path. |
ca_bundles |
Optional named list or character vector mapping feed IDs to
request-scoped CA bundle paths. When omitted, |
Summary data.table.
Synchronize All Registered SDMX Series
sync_all_sdmx_registry_data( registry = get_sdmx_registry(), local_path = NULL, ... )sync_all_sdmx_registry_data( registry = get_sdmx_registry(), local_path = NULL, ... )
registry |
Optional SDMX series registry. |
local_path |
Optional SDMX storage directory. |
... |
Passed to |
A standardized batch summary data.table.
Synchronize Registered SEC Company Facts
sync_all_sec_companyfacts_registry_data( registry = get_sec_registry(), config = NULL, local_path = NULL, ... )sync_all_sec_companyfacts_registry_data( registry = get_sec_registry(), config = NULL, local_path = NULL, ... )
registry |
Optional SEC company registry. |
config |
Optional SEC configuration. |
local_path |
Optional SEC Company Facts storage directory. |
... |
Passed to |
A standardized batch summary data.table.
Synchronize Registered SEC XBRL Frames
sync_all_sec_frames_registry_data( registry = get_sec_frames_registry(), config = NULL, local_path = NULL )sync_all_sec_frames_registry_data( registry = get_sec_frames_registry(), config = NULL, local_path = NULL )
registry |
Optional Frames registry. |
config |
Optional SEC configuration. |
local_path |
Optional frame storage directory. |
A standardized batch summary.
Synchronize Registered SEC Submissions
sync_all_sec_submissions_registry_data( registry = get_sec_registry(), config = NULL, local_path = NULL, ... )sync_all_sec_submissions_registry_data( registry = get_sec_registry(), config = NULL, local_path = NULL, ... )
registry |
Optional SEC company registry. |
config |
Optional SEC configuration. |
local_path |
Optional SEC submissions storage directory. |
... |
Passed to |
A standardized batch summary data.table.
Synchronize All Treasury Rate Datasets
sync_all_treasury_rates( datasets = names(.treasury_dataset_map()), years = NULL, local_path = NULL )sync_all_treasury_rates( datasets = names(.treasury_dataset_map()), years = NULL, local_path = NULL )
datasets |
Optional dataset vector. If omitted, sync all supported Treasury rate datasets. |
years |
Optional year filter passed to |
local_path |
Optional local storage path. |
Summary data.table.
Synchronize All World Bank Registry Series
sync_all_wbstats_registry_data( registry = get_wbstats_registry(), local_path = NULL, ... )sync_all_wbstats_registry_data( registry = get_wbstats_registry(), local_path = NULL, ... )
registry |
Optional World Bank registry table. |
local_path |
Optional local storage path. |
... |
Passed to |
Summary data.table.
Synchronize All Yahoo Finance Tickers In The Registry
sync_all_yahoofinance_registry_data( from = NULL, to = Sys.Date(), registry = get_yahoofinance_registry(), local_path = NULL, src = "yahoo", overlap_days = 10L, initial_lookback_days = 400L, max_attempts = 3L, retry_delay_seconds = 1, fallback_source = NULL, fallback_ticker = NULL )sync_all_yahoofinance_registry_data( from = NULL, to = Sys.Date(), registry = get_yahoofinance_registry(), local_path = NULL, src = "yahoo", overlap_days = 10L, initial_lookback_days = 400L, max_attempts = 3L, retry_delay_seconds = 1, fallback_source = NULL, fallback_ticker = NULL )
from |
Optional start date passed to |
to |
End date passed to |
registry |
Optional Yahoo Finance registry table. When omitted, the configured runtime registry is validated against the package seed before any provider request. |
local_path |
Optional local storage path. |
src |
quantmod source, default |
overlap_days |
Integer safety overlap used when deriving per-ticker incremental start dates from local data. |
initial_lookback_days |
Integer fallback lookback for tickers without
local data when |
max_attempts |
Maximum bounded attempts for a transient source failure. |
retry_delay_seconds |
Initial retry delay in seconds; delays use exponential backoff. |
fallback_source |
Optional explicitly configured fallback provider.
Currently supports |
fallback_ticker |
Optional provider-specific fallback identifier. |
Summary data.table.
Synchronize One Alpha Vantage Daily Series
sync_local_alphavantage_data( symbol, mode = NULL, config = NULL, local_path = NULL )sync_local_alphavantage_data( symbol, mode = NULL, config = NULL, local_path = NULL )
symbol |
A character string for the stock ticker symbol (e.g., |
mode |
Character string; either |
config |
A list of AlphaVantage API settings, typically retrieved via |
local_path |
Optional Alpha Vantage storage directory. |
A standard synchronization result.
Synchronize One BEA Regional Series
sync_local_bea_data( series_id, table_name, line_code, geofips = "STATE", year = "ALL", label = NULL, config = NULL, local_path = NULL )sync_local_bea_data( series_id, table_name, line_code, geofips = "STATE", year = "ALL", label = NULL, config = NULL, local_path = NULL )
series_id |
Stable local series identifier. |
table_name |
BEA Regional table name. |
line_code |
BEA line code. |
geofips |
Geography selector such as |
year |
Year selector, default |
label |
Optional local label. |
config |
Optional BEA configuration. |
local_path |
Optional BEA storage directory. |
A standard synchronization result.
Synchronize Local Binance Kline Data
sync_local_binance_klines( symbol = "ETHUSDT", interval = "1m", start_time = NULL, end_time = NULL, limit = 1500L, tz = "UTC", paginate = TRUE, local_path = NULL, storage = c("single", "monthly") )sync_local_binance_klines( symbol = "ETHUSDT", interval = "1m", start_time = NULL, end_time = NULL, limit = 1500L, tz = "UTC", paginate = TRUE, local_path = NULL, storage = c("single", "monthly") )
symbol |
Trading pair symbol. |
interval |
Candlestick interval. |
start_time |
Optional start time. |
end_time |
Optional end time. |
limit |
Page size. Binance futures allows up to 1500. |
tz |
Time zone applied to returned timestamps. |
paginate |
Logical. Request multiple pages when needed. |
local_path |
Optional Binance storage path. |
storage |
Local storage mode: monolithic |
A sync result list.
Synchronize One BLS Series
sync_local_bls_data( series_id, label = NULL, from = NULL, to = NULL, config = NULL, local_path = NULL, overlap_years = 2L )sync_local_bls_data( series_id, label = NULL, from = NULL, to = NULL, config = NULL, local_path = NULL, overlap_years = 2L )
series_id |
BLS series identifier. |
label |
Optional local label. |
from, to
|
Optional year or date bounds. |
config |
Optional BLS configuration. |
local_path |
Optional BLS storage directory. |
overlap_years |
Years re-fetched for revisions. |
A standard synchronization result.
Synchronize One Census Series
sync_local_census_data( series_id, dataset, data_type_code, category_code, seasonally_adj = "yes", from = NULL, to = NULL, label = NULL, config = NULL, local_path = NULL, overlap_months = 24L )sync_local_census_data( series_id, dataset, data_type_code, category_code, seasonally_adj = "yes", from = NULL, to = NULL, label = NULL, config = NULL, local_path = NULL, overlap_months = 24L )
series_id |
Stable local series identifier. |
dataset |
Economic Indicators dataset suffix, such as |
data_type_code |
Census item type code. |
category_code |
Census industry/category code. |
seasonally_adj |
Seasonal adjustment value. |
from, to
|
Optional time bounds. |
label |
Optional local label. |
config |
Optional Census configuration. |
local_path |
Optional Census storage directory. |
overlap_months |
Months re-fetched for revisions. |
A standard synchronization result.
Synchronize One CFTC COT Report
sync_local_cftc_cot( report_variant = c("futures_only", "combined"), report_type = "tff", report_id = NULL, dataset_id = NULL, market_codes = NULL, from = NULL, to = NULL, local_path = NULL, overlap_days = 14L, page_size = 5000L )sync_local_cftc_cot( report_variant = c("futures_only", "combined"), report_type = "tff", report_id = NULL, dataset_id = NULL, market_codes = NULL, from = NULL, to = NULL, local_path = NULL, overlap_days = 14L, page_size = 5000L )
report_variant |
Report variant: |
report_type |
Report family: |
report_id |
Stable local report identifier. |
dataset_id |
Optional official Socrata dataset identifier. |
market_codes |
Optional CFTC contract-market codes. |
from, to
|
Optional inclusive report-date bounds. |
local_path |
Optional CFTC storage directory. |
overlap_days |
Number of days re-fetched around the latest local report. |
page_size |
Number of source rows requested per page. |
A local synchronization result list.
Synchronize One Crypto Derivatives Dataset
sync_local_crypto_derivatives( provider, dataset_type, symbol, interval = NULL, from = NULL, to = NULL, limit = NULL, local_path = NULL, overlap_days = 2L, config = NULL, storage = c("single", "monthly") )sync_local_crypto_derivatives( provider, dataset_type, symbol, interval = NULL, from = NULL, to = NULL, limit = NULL, local_path = NULL, overlap_days = 2L, config = NULL, storage = c("single", "monthly") )
provider |
|
dataset_type |
Funding, open-interest, mark/index price, basis, or Binance long-short-ratio dataset identifier. |
symbol |
Provider instrument identifier. |
interval |
Required Binance open-interest period; otherwise a local label. |
from, to
|
Optional inclusive UTC time bounds. |
limit |
Provider page size. |
local_path |
Optional derivatives storage directory. |
overlap_days |
Days re-fetched before the latest local observation. |
config |
Optional exchange configuration. |
storage |
Local storage mode: monolithic |
A local synchronization result list.
Exchange REST APIs do not provide trustworthy market-wide liquidation history. This function persists events captured from Binance or OKX public liquidation streams without presenting private force-order history as market data.
sync_local_crypto_liquidations( events, provider, symbol, local_path = NULL, storage = c("single", "monthly") )sync_local_crypto_liquidations( events, provider, symbol, local_path = NULL, storage = c("single", "monthly") )
events |
Event rows containing a time column and price/quantity fields. |
provider |
Exchange identifier. |
symbol |
Exchange instrument identifier. |
local_path |
Optional derivatives storage directory. |
storage |
Local storage mode. |
A local synchronization result list.
Merges freshly retrieved source data into a local .rds file and stores a
sidecar metadata file containing the local update time and source update time.
sync_local_data( new_data, local_file_path, key_cols, order_cols = key_cols, source_utime = NULL, local_updated_at = Sys.time() )sync_local_data( new_data, local_file_path, key_cols, order_cols = key_cols, source_utime = NULL, local_updated_at = Sys.time() )
new_data |
A |
local_file_path |
Output |
key_cols |
Character vector of key columns used for de-duplication. |
order_cols |
Character vector used to sort the merged data. |
source_utime |
Optional upstream update time recorded in metadata. |
local_updated_at |
Optional local update time override. |
A list describing the sync result.
Combines multiple freshly retrieved data batches in memory, de-duplicates
them by key_cols, then calls sync_local_data() once. This is useful for
gap repair workflows where calling sync_local_data() once per small page
would repeatedly rewrite the same local .rds file.
sync_local_data_batches( batches, local_file_path, key_cols, order_cols = key_cols, source_utime = NULL, local_updated_at = Sys.time() )sync_local_data_batches( batches, local_file_path, key_cols, order_cols = key_cols, source_utime = NULL, local_updated_at = Sys.time() )
batches |
A list of |
local_file_path |
Output |
key_cols |
Character vector of key columns used for de-duplication. |
order_cols |
Character vector used to sort the merged data. |
source_utime |
Optional upstream update time recorded in metadata. |
local_updated_at |
Optional local update time override. |
A sync result list.
Upserts only monthly partitions touched by new_data. On the first
partitioned sync, an existing monolithic RDS cache is copied into monthly
partitions so callers can opt in without a separate migration step.
sync_local_data_partitioned( new_data, local_file_path, time_col, key_cols, order_cols = key_cols, source_utime = NULL, local_updated_at = Sys.time(), tz = "UTC" )sync_local_data_partitioned( new_data, local_file_path, time_col, key_cols, order_cols = key_cols, source_utime = NULL, local_updated_at = Sys.time(), tz = "UTC" )
new_data |
A |
local_file_path |
Output |
time_col |
Name of the timestamp column used for monthly partitioning. |
key_cols |
Character vector of key columns used for de-duplication. |
order_cols |
Character vector used to sort the merged data. |
source_utime |
Optional upstream update time recorded in metadata. |
local_updated_at |
Optional local update time override. |
tz |
Time zone used to determine month boundaries. |
A sync result list. Its data element contains the touched rows,
rather than the complete potentially large dataset.
Synchronize One EIA Series
sync_local_eia_data( series_id, label = NULL, config = NULL, from = NULL, to = NULL, local_path = NULL, overlap_days = 31L, page_size = 5000L )sync_local_eia_data( series_id, label = NULL, config = NULL, from = NULL, to = NULL, local_path = NULL, overlap_days = 31L, page_size = 5000L )
series_id |
EIA series identifier. |
label |
Optional local label. |
config |
Optional EIA API configuration. |
from, to
|
Optional inclusive period bounds. |
local_path |
Optional EIA storage directory. |
overlap_days |
Days re-fetched around the latest local observation. |
page_size |
Number of observations requested per page. |
A local synchronization result list.
Synchronize One Treasury Fiscal Data Table
sync_local_fiscaldata( dataset_id, endpoint, date_col = "record_date", key_cols = date_col, fields = NULL, from = NULL, to = NULL, local_path = NULL, overlap_days = 14L, page_size = 10000L )sync_local_fiscaldata( dataset_id, endpoint, date_col = "record_date", key_cols = date_col, fields = NULL, from = NULL, to = NULL, local_path = NULL, overlap_days = 14L, page_size = 10000L )
dataset_id |
Stable local dataset identifier. |
endpoint |
Fiscal Data endpoint relative to the fiscal-service base URL. |
date_col |
Source date column. |
key_cols |
Source columns that uniquely identify observations per date. |
fields |
Optional source fields to request. |
from, to
|
Optional inclusive date bounds. |
local_path |
Optional Fiscal Data storage directory. |
overlap_days |
Days re-fetched around the latest local date. |
page_size |
Number of source rows requested per page. |
A local synchronization result list.
Synchronize Local FRED Data
sync_local_fred_data( series_id, config = NULL, local_path = NULL, from_server = FALSE, tz = "America/Chicago" )sync_local_fred_data( series_id, config = NULL, local_path = NULL, from_server = FALSE, tz = "America/Chicago" )
series_id |
FRED series identifier. |
config |
Optional FRED API config. |
local_path |
Optional local storage path. |
from_server |
Logical. If |
tz |
Time zone used for source update time inference. |
A sync result list.
Synchronize Provider-Neutral Fund Holdings
sync_local_fund_holdings( data, provider, fund_id, local_path = NULL, as_of_date = NULL, column_map = list(), source_utime = NULL )sync_local_fund_holdings( data, provider, fund_id, local_path = NULL, as_of_date = NULL, column_map = list(), source_utime = NULL )
data |
Holdings table, standardized internally. |
provider |
Fund issuer/provider identifier. |
fund_id |
Fund ticker or stable identifier. |
local_path |
Optional generic holdings storage root. |
as_of_date |
Optional snapshot-date override. |
column_map |
Optional canonical-to-source column mapping. |
source_utime |
Optional upstream update time. |
A standard synchronization result.
Synchronize Local OKX Candle Data
sync_local_okx_candle( inst_id, bar, config = NULL, local_path = NULL, mode = c("latest", "history"), before = NULL, limit = 100L, tz = "UTC", storage = c("single", "monthly") )sync_local_okx_candle( inst_id, bar, config = NULL, local_path = NULL, mode = c("latest", "history"), before = NULL, limit = 100L, tz = "UTC", storage = c("single", "monthly") )
inst_id |
Instrument identifier. |
bar |
Candle interval. |
config |
Optional OKX API config. If omitted, defaults from the
package config and |
local_path |
Optional OKX storage path. |
mode |
Either |
before |
Optional history cursor. |
limit |
Integer page size. |
tz |
Output time zone. |
storage |
Local storage mode: monolithic |
A sync result list.
Synchronize Local quantmod OHLC Data
sync_local_quantmod_OHLC( ticker, label = ticker, from, to, src = "yahoo", local_path = NULL, max_attempts = 3L, retry_delay_seconds = 1, fallback_source = NULL, fallback_ticker = ticker )sync_local_quantmod_OHLC( ticker, label = ticker, from, to, src = "yahoo", local_path = NULL, max_attempts = 3L, retry_delay_seconds = 1, fallback_source = NULL, fallback_ticker = ticker )
ticker |
Market symbol passed to |
label |
Optional label to store in the standardized |
from |
Start date. |
to |
End date. |
src |
quantmod source, default |
local_path |
Optional local storage path. |
max_attempts |
Maximum bounded attempts for a transient source failure. |
retry_delay_seconds |
Initial retry delay in seconds; delays use exponential backoff. |
fallback_source |
Optional explicitly configured fallback provider.
Currently supports |
fallback_ticker |
Optional provider-specific fallback identifier. |
An external fallback never replaces a finite existing primary-source OHLC bar. It only fills keys whose local bars are missing or invalid.
A sync result list.
Synchronize Local RSS Data
sync_local_rss_data( feed_id, url, parser = c("plain", "gdpnow"), local_path = NULL, ca_bundle = NULL )sync_local_rss_data( feed_id, url, parser = c("plain", "gdpnow"), local_path = NULL, ca_bundle = NULL )
feed_id |
Local feed identifier. |
url |
RSS feed URL. |
parser |
Parsing strategy. Currently supports |
local_path |
Optional local storage path. |
ca_bundle |
Optional CA bundle path used only for this feed request. |
A sync result list.
Synchronize One SDMX Registry Entry
sync_local_sdmx_data( series_id, provider, base_url, agency = NULL, dataflow, version = "latest", key = "", flow_ref = NULL, format = NULL, accept = "text/csv", time_col = "TIME_PERIOD", value_col = "OBS_VALUE", dimension_cols = NULL, label = NULL, frequency = NULL, from = NULL, to = NULL, local_path = NULL, overlap_days = 62L )sync_local_sdmx_data( series_id, provider, base_url, agency = NULL, dataflow, version = "latest", key = "", flow_ref = NULL, format = NULL, accept = "text/csv", time_col = "TIME_PERIOD", value_col = "OBS_VALUE", dimension_cols = NULL, label = NULL, frequency = NULL, from = NULL, to = NULL, local_path = NULL, overlap_days = 62L )
series_id |
Stable local series identifier. |
provider |
Provider dialect: |
base_url |
Provider REST base URL. |
agency |
Optional SDMX agency identifier. |
dataflow |
SDMX dataflow identifier. |
version |
SDMX dataflow version. |
key |
SDMX series key. |
flow_ref |
Optional provider-specific flow reference. |
format |
Provider query format. |
accept |
HTTP response media type. |
time_col, value_col
|
Source observation columns. |
dimension_cols |
Columns identifying distinct series in the response. |
label, frequency
|
Optional local metadata. |
from, to
|
Optional inclusive SDMX periods. |
local_path |
Optional SDMX storage directory. |
overlap_days |
Days re-fetched around the latest local period. |
A local synchronization result list.
Download An Optional SEC Bulk Archive
sync_local_sec_bulk_archive( archive = c("companyfacts", "submissions"), local_path = NULL, config = NULL, overwrite = FALSE, extract = FALSE )sync_local_sec_bulk_archive( archive = c("companyfacts", "submissions"), local_path = NULL, config = NULL, overwrite = FALSE, extract = FALSE )
archive |
One of |
local_path |
Optional SEC bulk archive directory. |
config |
Optional SEC configuration. |
overwrite |
Replace the existing ZIP. |
extract |
Extract the ZIP into a same-named directory. |
Download metadata.
Synchronize SEC Company Facts For One Company
sync_local_sec_companyfacts( cik, ticker = NULL, company_name = NULL, concepts = NULL, forms = NULL, from = NULL, to = NULL, config = NULL, local_path = NULL, overlap_days = 31L )sync_local_sec_companyfacts( cik, ticker = NULL, company_name = NULL, concepts = NULL, forms = NULL, from = NULL, to = NULL, config = NULL, local_path = NULL, overlap_days = 31L )
cik |
SEC Central Index Key. |
ticker |
Optional ticker label. |
company_name |
Optional company name label. |
concepts |
Optional concept names or |
forms |
Optional filing forms to retain. |
from, to
|
Optional inclusive filed-date bounds. |
config |
Optional SEC configuration. |
local_path |
Optional SEC Company Facts storage directory. |
overlap_days |
Filed-date overlap used for local upserts. |
A local synchronization result list.
Download A Selected SEC Filing Document
sync_local_sec_filing_document( cik, accession_number, primary_document, local_path = NULL, config = NULL, overwrite = FALSE )sync_local_sec_filing_document( cik, accession_number, primary_document, local_path = NULL, config = NULL, overwrite = FALSE )
cik |
SEC Central Index Key. |
accession_number |
Filing accession number. |
primary_document |
Primary filing document name from submissions data. |
local_path |
Optional document cache directory. |
config |
Optional SEC configuration. |
overwrite |
Replace an existing local document. |
A one-row metadata data.table.
Synchronize One SEC XBRL Frame
sync_local_sec_frame( taxonomy, tag, unit, period, config = NULL, local_path = NULL )sync_local_sec_frame( taxonomy, tag, unit, period, config = NULL, local_path = NULL )
taxonomy |
XBRL taxonomy, such as |
tag |
XBRL concept tag. |
unit |
XBRL unit, such as |
period |
SEC frame period, such as |
config |
Optional SEC configuration. |
local_path |
Optional SEC frame cache directory. |
A standard synchronization result.
Synchronize SEC Submissions For One Company
sync_local_sec_submissions( cik, ticker = NULL, company_name = NULL, forms = NULL, from = NULL, to = NULL, include_history = TRUE, config = NULL, local_path = NULL, overlap_days = 14L )sync_local_sec_submissions( cik, ticker = NULL, company_name = NULL, forms = NULL, from = NULL, to = NULL, include_history = TRUE, config = NULL, local_path = NULL, overlap_days = 14L )
cik |
SEC Central Index Key. |
ticker |
Optional ticker label. |
company_name |
Optional company name label. |
forms |
Optional filing forms to retain. |
from, to
|
Optional inclusive filing-date bounds. |
include_history |
Logical; retrieve older SEC submission files in addition to recent filings. |
config |
Optional SEC configuration. |
local_path |
Optional SEC submissions storage directory. |
overlap_days |
Filing-date overlap for incremental synchronization. |
A local synchronization result list.
Synchronize Local Treasury Rates
sync_local_treasury_rates(dataset, years = NULL, local_path = NULL)sync_local_treasury_rates(dataset, years = NULL, local_path = NULL)
dataset |
Treasury dataset key. Supported values are |
years |
Optional integer vector of calendar years to fetch. If omitted, the function combines the historical archive with current-year annual XML feeds from 2024 through the current year. |
local_path |
Optional local storage path. |
A sync result list.
Synchronize Local World Bank Data
sync_local_wbstats_data(indicator, country, freq = "Y", local_path = NULL, ...)sync_local_wbstats_data(indicator, country, freq = "Y", local_path = NULL, ...)
indicator |
World Bank indicator code. |
country |
Country or aggregate code. |
freq |
Frequency code. |
local_path |
Optional local storage path. |
... |
Passed to |
A sync result list.
Download Selected Documents From Local SEC Submissions
sync_sec_filing_documents( cik, forms = NULL, from = NULL, to = NULL, limit = 20L, submissions_path = NULL, local_path = NULL, config = NULL )sync_sec_filing_documents( cik, forms = NULL, from = NULL, to = NULL, limit = 20L, submissions_path = NULL, local_path = NULL, config = NULL )
cik |
SEC Central Index Key. |
forms |
Optional filing-form filter. |
from, to
|
Optional filing-date bounds. |
limit |
Maximum number of newest documents. |
submissions_path |
Optional submissions cache directory. |
local_path |
Optional document cache directory. |
config |
Optional SEC configuration. |
One metadata row per selected document.
Validates the package-owned, provider-neutral instrument catalog and its Yahoo identifier and fallback projection against a Yahoo Finance registry. The Yahoo registry remains the sync authority: catalog fallback entries are validated projections, not instructions for synchronization.
validate_instrument_catalog( catalog_path = .instrument_catalog_seed_path(), yahoo_registry_path = get_yahoofinance_seed_registry_path() )validate_instrument_catalog( catalog_path = .instrument_catalog_seed_path(), yahoo_registry_path = get_yahoofinance_seed_registry_path() )
catalog_path |
Path to a catalog JSON file. Defaults to the packaged seed. |
yahoo_registry_path |
Path to the Yahoo Finance sync registry. Defaults to the packaged seed registry so the catalog is usable without a runtime configuration file. Pass a runtime registry path to validate that runtime registry explicitly. |
Every public field is required and non-null except that fallback_sources
may be an empty array. asset_class must be one of equity,
fixed_income, commodity, foreign_exchange, or cryptocurrency.
market_calendar is the completed-bar session calendar, not a listing
venue. Schema 1.0.0 accepts legacy XNYS as a calendar alias; schema
1.1.0 uses values such as US_EQUITY, XSHG, FX_24_5, and
CRYPTO_24_7. provider_identifiers is a non-empty named list whose
values are non-empty provider symbols. quote_currency is a three-letter
uppercase currency code, or the recognized stablecoin quote code USDT or
USDC; CNH is permitted as the conventional offshore Chinese-yuan quote
code. price_frequency and every supported_intervals entry are limited
to 1d or 4h and must match
price_frequency. Perpetual swaps must provide an OKX identifier and
positive contract_size and quantity_step, their units, settlement
currency, and contract_structure = "linear_usdt_margined_perpetual".
Each fallback_sources object has exactly provider and symbol fields;
pairs must be unique and ordered lexicographically by provider then symbol.
A list with valid, an errors data table, and a catalog data
table. The catalog retains nested provider mappings as list columns.
Validates the provider-neutral catalog and returns its active or inactive
4-hour OKX perpetual-swap subset. When local_path is supplied, every
active instrument must have a non-empty local cache with a finite completed
OHLC row. OKX cache ingestion already excludes unconfirmed candles, so this
check never treats an in-progress candle as usable.
validate_okx_perpetual_catalog( catalog_path = .instrument_catalog_seed_path(), local_path = NULL, as_of = Sys.time() )validate_okx_perpetual_catalog( catalog_path = .instrument_catalog_seed_path(), local_path = NULL, as_of = Sys.time() )
catalog_path |
Path to a catalog JSON file. Defaults to the packaged seed. |
local_path |
Optional local OKX candle-cache directory. Supplying it enables catalog-to-cache validation. |
as_of |
UTC time used to exclude cache rows at or after the cutoff. |
A list with valid, errors, catalog, and cache_status data
tables.
Compare required ticker entries and fallback declarations in a runtime
registry with the package-managed seed. Additional runtime-only ticker
metadata is permitted. Seed entries with required: true must be present;
every seed fallback declaration must be present unchanged. Missing seed-owned
degraded-cache policy fields are projected by get_yahoofinance_registry();
explicitly weaker supplied values are invalid.
validate_yahoofinance_registry( registry_path = get_yahoofinance_registry_file_path(), seed_path = get_yahoofinance_seed_registry_path() )validate_yahoofinance_registry( registry_path = get_yahoofinance_registry_file_path(), seed_path = get_yahoofinance_seed_registry_path() )
registry_path |
Runtime registry JSON path. |
seed_path |
Package seed registry JSON path. |
A list with valid, missing, mismatched, and
missing_required data tables.