Go to file
Amir Alexander Abdelbaki d52cb8b12e Initial commit: NewsAtlas news globe
nginx-fronted stack that pulls RSS from an ideologically mixed set of
outlets, geocodes stories onto a 3D globe with zoom-adaptive clustering,
tracks market/oil prices with spike-to-article correlation, and overlays
weather, conflict events, Wikipedia lookups, and parliament diagrams.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Um48tTvZDrEgDeweFyhPYC
2026-07-20 13:49:00 +02:00
backend Initial commit: NewsAtlas news globe 2026-07-20 13:49:00 +02:00
frontend Initial commit: NewsAtlas news globe 2026-07-20 13:49:00 +02:00
nginx Initial commit: NewsAtlas news globe 2026-07-20 13:49:00 +02:00
.env.example Initial commit: NewsAtlas news globe 2026-07-20 13:49:00 +02:00
.gitignore Initial commit: NewsAtlas news globe 2026-07-20 13:49:00 +02:00
README.md Initial commit: NewsAtlas news globe 2026-07-20 13:49:00 +02:00
docker-compose.yml Initial commit: NewsAtlas news globe 2026-07-20 13:49:00 +02:00

README.md

NewsAtlas

A self-hosted, nginx-fronted "news globe": pulls RSS from a deliberately ideologically-mixed set of outlets, localizes each story onto a 3D globe, merges same-place/same-topic stories into a single expandable point, tracks major stock indices and oil prices over time, and overlays weather and (where available) conflict-event data. Any place name or article text can be looked up on Wikipedia, and country panels show that country's parliament composition diagram when Wikipedia has one.

Layout

  • Left rail — latest news, newest first, auto-refreshing. A 📍 button on any geocoded story flies the globe to it.
  • Right rail — index/oil ticker. Click an instrument to expand its recent-spikes-with-candidate-articles view (see below).
  • Center — the globe. Clicking a point opens a modal with its stories, a Wikipedia summary of the place, and (if available) its country's parliament composition diagram. Text selected anywhere can be searched on Wikipedia via the popup that appears.
  • Bottom drawer — collapsible conflict/military-event log (ACLED-backed; see below).

Globe points use zoom-adaptive clustering: the backend groups articles onto a fixed ~11km grid, then the frontend progressively folds nearby grid cells into a single point as you zoom out (regroupForAltitude in frontend/js/app.js), so a continent-level view doesn't leave dozens of separate dots. The fold radius grows with camera altitude, capped at subcontinent scale — tune it via mergeRadiusDeg() if you want tighter or looser regional grouping.

Color theme is the user's own "CyberQueer" palette (near-black base, hot pink #E40046 + electric violet #5018DD accents), including a tinted globe material — see frontend/css/style.css :root variables to swap it.

National borders render as a bright violet stroke over a transparent fill (Natural Earth admin-0 polygons, loaded from three-globe's own npm package via unpkg — no key or backend endpoint needed).

Each market instrument in the right rail shows a 7-day sparkline (line + area, hover for a crosshair/tooltip) built from /api/markets/history. Article listings (news feed, cluster modal, spike candidate articles) show a small favicon next to each headline, resolved from the article's own URL — decorative only, not a claim of any publisher's official branding.

Stack

  • backend/ — FastAPI + SQLite + APScheduler. Polls RSS feeds, geocodes articles against a curated gazetteer, clusters them, polls Yahoo Finance for markets/oil, optionally polls ACLED for conflict events, and proxies Wikipedia + OpenWeatherMap so API keys never reach the browser.
  • frontend/ — static HTML/CSS/JS using globe.gl (three.js) for the 3D globe. No build step.
  • nginx/ — reverse proxy (/api/* → backend) + static file server, with a short-lived cache in front of the API so many browser tabs don't hammer SQLite.

Running it

cp .env.example .env   # fill in optional API keys, see below
docker compose up --build

Then open http://localhost:8080 (or whatever HTTP_PORT you set).

On first boot the globe will be empty for a few seconds until the first RSS poll completes — hit "Refresh now" if you don't want to wait for the 10-minute interval.

Optional API keys (.env)

Everything works with .env left blank except that the weather and conflict overlays disable themselves. Nothing else requires a key.

Var What it unlocks Get one
OWM_API_KEY Weather overlay (clouds/precipitation/temp/wind/pressure) free, openweathermap.org
ACLED_API_KEY + ACLED_EMAIL Conflict/"military movement" event overlay free for registered (incl. non-commercial/research) use, acleddata.com

Why ACLED for "military movements": there is no single free, public, real-time feed of military movements. ACLED is the closest widely-used open dataset of sourced, georeferenced conflict/political-violence events and is what the conflict overlay is built on. Treat it as "reported conflict events," not live troop tracking.

What's a heuristic, not ground truth

  • Geocoding (backend/app/geocode.py, data/gazetteer.csv) is keyword matching against a curated ~180-place gazetteer, not full NLP/NER. It will miss unlisted places and can mismatch generic names. Add rows to the CSV to extend coverage.
  • Clustering groups articles whose matched location falls in the same ~11km grid cell within the last ARTICLE_WINDOW_HOURS (default 72h). The "topic" label shown per cluster is just the most frequent significant words across its headlines, not topic modeling.
  • Weather overlay stitches a handful of OpenWeatherMap Web-Mercator tiles into one texture and maps it onto the globe's equirectangular UVs. It's visually indicative (clouds/precip patterns are recognizable) but not pixel-accurate, especially near the poles.
  • Parliament diagrams (backend/app/wikipedia.py, data/parliaments.yaml) reuse the infobox image of each country's legislature article, which is usually — not always — the semicircle/ hemicycle composition chart. Countries without a curated mapping are resolved by a live Wikipedia search at request time, so accuracy varies; fix a bad match by adding an override to parliaments.yaml.
  • MOEX (Russia) index data from Yahoo Finance appears frozen at mid-2022 values — Western data providers largely cut off live Russian market feeds after sanctions. It's shown for continuity, not as a live price.
  • RSS feed URLs (backend/app/sources.yaml) belong to third parties and can change without notice — if a source goes quiet, check its site for a current feed URL and update the file (no rebuild needed, it's a read-only mount).

API surface (consumed by the frontend, but usable standalone)

  • GET /api/clusters — merged globe points
  • GET /api/clusters/{cluster_key}/articles — articles behind one point
  • GET /api/articles?q=&limit= — raw article search
  • POST /api/refresh — force an immediate RSS poll
  • GET /api/markets/latest, GET /api/markets/history?symbol=^GSPC
  • GET /api/markets/spikes?symbol=&hours= — sudden index/oil moves (MARKET_SPIKE_THRESHOLD_PCT) paired with relevance-ranked articles published in the surrounding window (MARKET_SPIKE_LOOKBACK_HOURS) — click a ticker item in the UI for this. It's a heuristic correlation (keyword + country/location matching), not a verified causal link.
  • GET /api/conflict-events?hours=168
  • GET /api/weather/tiles/{layer}/{z}/{x}/{y}.png — OWM tile proxy
  • GET /api/wikipedia/summary?title=, GET /api/wikipedia/search?q=, GET /api/wikipedia/parliament?country=

Extending

  • Add/edit RSS sources: backend/app/sources.yaml.
  • Add gazetteer locations: backend/app/data/gazetteer.csv.
  • Add/fix parliament page mappings: backend/app/data/parliaments.yaml.
  • Change poll intervals / clustering window: .env.