docs: Rewrite documentation with accurate project state

- Delete obsolete change proposals and bio content source
- Rewrite README.md with correct features, data sources, structure
- Update PROJECT_REFERENCE.md with accurate status and completed work
- Update CLAUDE.md references and sprint status
- Add docs/CONTRIBUTING.md developer guide with:
  - How to add blog posts (frontmatter, markdown)
  - How to add new pages (Dash routing)
  - How to add dashboard tabs
  - How to create figure factories
  - Branch workflow and code standards

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
2026-01-17 12:27:25 -05:00
parent 1a878313f8
commit 4818c53fd2
7 changed files with 794 additions and 1190 deletions

139
README.md
View File

@@ -1,36 +1,42 @@
# Analytics Portfolio
A data analytics portfolio showcasing end-to-end data engineering, visualization, and analysis capabilities.
A personal portfolio website showcasing data engineering and visualization capabilities, featuring an interactive Toronto Neighbourhood Dashboard.
## Projects
## Live Pages
### Toronto Housing Dashboard
| Route | Page | Description |
|-------|------|-------------|
| `/` | Home | Bio landing page |
| `/about` | About | Background and experience |
| `/projects` | Projects | Portfolio project showcase |
| `/resume` | Resume | Professional CV |
| `/contact` | Contact | Contact form |
| `/blog` | Blog | Technical articles |
| `/blog/{slug}` | Article | Individual blog posts |
| `/toronto` | Toronto Dashboard | Neighbourhood analysis (5 tabs) |
| `/toronto/methodology` | Methodology | Dashboard data sources and methods |
| `/health` | Health | API health check endpoint |
An interactive choropleth dashboard analyzing Toronto's housing market using multi-source data integration.
## Toronto Neighbourhood Dashboard
**Features:**
- Purchase market analysis from TRREB monthly reports
- Rental market analysis from CMHC annual surveys
- Interactive choropleth maps by district/zone
- Time series visualization with policy event annotations
- Purchase/Rental mode toggle
An interactive choropleth dashboard analyzing Toronto's 158 official neighbourhoods across five dimensions:
**Data Sources:**
- [TRREB Market Watch](https://trreb.ca/market-data/market-watch/) - Monthly purchase statistics
- [CMHC Rental Market Survey](https://www.cmhc-schl.gc.ca/professionals/housing-markets-data-and-research/housing-data/data-tables/rental-market) - Annual rental data
- **Overview**: Composite livability scores, income vs safety scatter
- **Housing**: Affordability index, rent trends, dwelling types
- **Safety**: Crime rates, breakdowns by type, trend analysis
- **Demographics**: Income distribution, age pyramids, population density
- **Amenities**: Parks, schools, transit accessibility
**Tech Stack:**
- Python 3.11+ / Dash / Plotly
- PostgreSQL + PostGIS
- dbt for data transformation
- Pydantic for validation
- SQLAlchemy 2.0
**Data Sources**:
- City of Toronto Open Data Portal (neighbourhoods, census profiles, amenities)
- Toronto Police Service (crime statistics)
- CMHC Rental Market Survey (rental data by zone)
## Quick Start
```bash
# Clone and setup
git clone https://github.com/lmiranda/personal-portfolio.git
git clone https://gitea.hotserv.cloud/lmiranda/personal-portfolio.git
cd personal-portfolio
# Install dependencies and configure environment
@@ -55,48 +61,72 @@ portfolio_app/
├── app.py # Dash app factory
├── config.py # Pydantic settings
├── pages/
│ ├── home.py # Bio landing page (/)
── toronto/ # Toronto dashboard (/toronto)
│ ├── home.py # Bio landing (/)
── about.py # About page
│ ├── contact.py # Contact form
│ ├── projects.py # Project showcase
│ ├── resume.py # Resume/CV
│ ├── blog/ # Blog system
│ │ ├── index.py # Article listing
│ │ └── article.py # Article renderer
│ └── toronto/ # Toronto dashboard
│ ├── dashboard.py # Main layout with tabs
│ ├── methodology.py # Data documentation
│ ├── tabs/ # Tab layouts (5)
│ └── callbacks/ # Interaction logic
├── components/ # Shared UI components
├── figures/ # Plotly figure factories
── toronto/ # Toronto data logic
── parsers/ # PDF/CSV extraction
├── loaders/ # Database operations
├── schemas/ # Pydantic models
── models/ # SQLAlchemy ORM
── content/
── blog/ # Markdown blog articles
├── toronto/ # Toronto data logic
├── parsers/ # API data extraction
── loaders/ # Database operations
│ ├── schemas/ # Pydantic models
│ └── models/ # SQLAlchemy ORM
└── errors/ # Exception handling
dbt/
├── models/
│ ├── staging/ # 1:1 source tables
│ ├── intermediate/ # Business logic
│ └── marts/ # Analytical tables
│ ├── staging/ # 1:1 source tables
│ ├── intermediate/ # Business logic
│ └── marts/ # Analytical tables
notebooks/ # Data documentation (15 notebooks)
├── overview/ # Overview tab visualizations
├── housing/ # Housing tab visualizations
├── safety/ # Safety tab visualizations
├── demographics/ # Demographics tab visualizations
└── amenities/ # Amenities tab visualizations
docs/
├── PROJECT_REFERENCE.md # Architecture reference
├── CONTRIBUTING.md # Developer guide
└── project-lessons-learned/
```
## Tech Stack
| Layer | Technology |
|-------|------------|
| Database | PostgreSQL 16 + PostGIS |
| Validation | Pydantic 2.x |
| ORM | SQLAlchemy 2.x |
| Transformation | dbt-postgres |
| Data Processing | Pandas, GeoPandas |
| Visualization | Dash + Plotly |
| UI Components | dash-mantine-components |
| Testing | pytest |
| Python | 3.11+ |
## Development
```bash
make test # Run tests
make lint # Run linter
make test # Run pytest
make lint # Run ruff linter
make format # Format code
make ci # Run all checks
```
## Data Pipeline
```
Raw Files (PDF/Excel)
Parsers (pdfplumber, pandas)
Pydantic Validation
SQLAlchemy Loaders
PostgreSQL + PostGIS
dbt Transformations
Dash Visualization
make dbt-run # Run dbt models
make dbt-test # Run dbt tests
```
## Environment Variables
@@ -109,12 +139,19 @@ POSTGRES_USER=portfolio
POSTGRES_PASSWORD=<secure>
POSTGRES_DB=portfolio
DASH_DEBUG=true
SECRET_KEY=<random>
```
## Documentation
- **For developers**: See `docs/CONTRIBUTING.md` for setup and contribution guidelines
- **For Claude Code**: See `CLAUDE.md` for AI assistant context
- **Architecture**: See `docs/PROJECT_REFERENCE.md` for technical details
## License
MIT
## Author
Leo Miranda - [GitHub](https://github.com/lmiranda) | [LinkedIn](https://linkedin.com/in/yourprofile)
Leo Miranda