EVAN / SUN Résumé
Research & project showcases

03 · Market data engineering

Stock Market & Cryptocurrency Prediction

A cross-asset data pipeline that normalizes digital-asset, equity-index, commodity, and gold history for exploratory predictive analysis.

cross-asset signal setMARKET DATA
CRYPTO×INDEX×COMMODITY

normalize · store · engineer features

Role

Data engineering and quantitative research - market-data normalization, SQLite design, feature engineering, and documentation.

Context

Digital assets trade continuously while equity indices use market-hours calendars, making a consistent cross-asset dataset a non-trivial preparation problem.

Tools

Python · SQLite · pandas · NumPy · Yahoo Finance data · Feature engineering

Project overview

The project turns heterogeneous daily market CSVs into a shared SQLite schema and builds return, trend, and volatility features for each asset.

Research question

How can heterogeneous market histories be cleaned, stored, and made comparable before testing cross-asset predictive signals?

Workflow

  1. 01Normalize raw columns and date formats across daily asset histories.
  2. 02Load cleaned series into SQLite tables for asset prices and engineered features.
  3. 03Generate one-day returns, log returns, moving averages, and seven-day volatility for each asset.
  4. 04Use the structured dataset as a documented foundation for cross-asset modeling rather than claiming a final trading signal.
02

Data & methodology

Data sources

  • Local five-year CSV histories for ETH, SOL, S&P 500, Nasdaq, crude oil, and gold
  • Source report documenting Yahoo Finance historical-data collection
  • SQLite market-data database

Methods

  • Schema normalization
  • Date and numeric cleaning
  • SQLite persistence
  • Return, moving-average, and volatility feature engineering
Evidence boundary
Input→Preparation→Analysis→Output

The showcase is based on local scripts, CSV inventory, SQLite schema, and report documentation; raw market data is not published.

03

Selected code

python

Per-asset feature construction

df["return_1d"] = df["close"].pct_change()
df["log_return"] = np.log(df["close"] / df["close"].shift(1))
df["ma_5"] = df["close"].rolling(5).mean()
df["ma_10"] = df["close"].rolling(10).mean()
df["volatility_7d"] = df["return_1d"].rolling(7).std()

The feature builder applies the same documented daily transformations to each normalized asset series before persisting the result.

Results & conclusion

Key outputs

  • asset_prices SQLite table
  • asset_features SQLite table
  • Repeatable Python loaders
  • Analysis-ready daily feature dataset

Documented findings

  • The pipeline creates a common daily schema across assets with different source column conventions.
  • Feature generation is reproducible and separated from raw-data loading.

Conclusion

The strongest deliverable is a reusable cross-asset research foundation; prediction performance is not asserted without a documented final model evaluation.

Decision relevance

  • A shared, auditable data layer makes subsequent correlation, lag, and forecasting work more reliable.
  • Normalization is especially important when combining 24/7 crypto series with market-hours assets.

Limitations

  • A common schema does not by itself establish predictive power.
  • Different trading calendars and missing values require careful alignment in downstream models.
  • No public live demo is claimed.
Next project showcase
04 · Event-driven data engineering

Kafka Crypto-Trade Streaming

View all research & project showcases