SpaceNews

SpaceNews is a web application that aggregates and processes news articles related to space exploration and astronomy. The project takes advantage of a news API that provides links to relevant websites. Once these links are obtained, a custom scraper navigates through each site to extract the content.

Website

Project Description - SpaceNews

SpaceNews is a web application that aggregates and processes news articles related to space exploration and astronomy. The project takes advantage of a news API that provides links to relevant websites. Once these links are obtained, a custom scraper navigates through each site to extract the content.

The extracted content is then processed by a Large Language Model (LLM) which generates concise summaries, tags, and additional metadata. This processed information is stored in a database and can be accessed either through a React frontend or an API.

Technologies Used

  • Frontend: React
  • Backend: Custom scraper, LLM processing
  • Storage: Database for storing processed content
  • Languages: JavaScript, TypeScript
  • APIs: News API for retrieving article links

Features

  • News Aggregation: Retrieves links to news articles from a dedicated API.
  • Content Scraping: Custom scraper extracts content from each linked website.
  • LLM Processing: Uses a Large Language Model to create summaries, tags, and additional metadata.
  • Data Storage: Processed information is stored in a database for easy access.
  • Multiple Access Points: Users can access the summarized news via a React frontend or through an API.

Workflow

  1. API Integration: Fetch links to space-related news articles.
  2. Scraping: Navigate through each link to extract the full article content.
  3. Processing: Generate summaries and tags using an LLM.
  4. Storage: Save the processed content in a database.
  5. Access: Provide access to the stored information through a React frontend or an API.