Tag
#data-pipeline
13 repositories
Repos
Current working version of the Data Aggregator + Analyst agent in one solution via the CLI
⚡ High-performance async web crawler & data ingestion framework in Rust — Bloom dedup, proxy rotation, distributed workers, Solana indexing, Parquet/ClickHouse export. 10k+ pages/sec.
Due Diligence Automation for Crypto Funds is an AI tool from ESPRIT University that automates digital asset evaluations using GPT, web scraping, and financial data. It quickly generates smart questions, gathers data, and creates dynamic reports to help investors make informed decisions.
An end-to-end data engineering project and analytical application designed to support long-term Bitcoin investing. Built with dbt, Snowflake, Airflow, and Streamlit.
Production-ready Solana Web3 indexer in Go with PostgreSQL, real-time WebSocket APIs, and Prometheus metrics for scalable on-chain data pipelines.
Institutional-grade Python Data Pipeline for real-time Ethereum On-Chain Risk Monitoring. Built for Crypto Funds.
Go tool to download and aggregate Binance aggTrades into hourly bars with order flow metrics and adaptive whale detection (rolling P99/P99.9). Designed for neural network training with Parquet output.
A minimal ETL (Extract, Transform, Load) pipeline written in Rust for fetching and processing block data from the Solana blockchain.
Production crypto market data pipeline for collecting, cleaning, and storing multi-exchange OHLCV time series.
📊 Extract 30+ trading metrics (CVD, VWAP, Imbalance, Exhaustion, Large Trades) from Bybit BTC/USDT order book and trade data. Built with Polars for 10x speed. Kaggle-ready pipeline processes 245 days in 4-5 hours.
Real-time blockchain analytics engine for transaction monitoring, anomaly detection, and risk scoring | Data pipeline
🌐 Preserve Solana projects effortlessly with Winmem, a self-hostable AI runtime that maintains verifiable records even when teams step away.