Project Description - SpaceNews
SpaceNews is a web application that aggregates and processes news articles related to space exploration and astronomy. The project takes advantage of a news API that provides links to relevant websites. Once these links are obtained, a custom scraper navigates through each site to extract the content.
The extracted content is then processed by a Large Language Model (LLM) which generates concise summaries, tags, and additional metadata. This processed information is stored in a database and can be accessed either through a React frontend or an API.
Technologies Used
- Frontend: React
- Backend: Custom scraper, LLM processing
- Storage: Database for storing processed content
- Languages: JavaScript, TypeScript
- APIs: News API for retrieving article links
Features
- News Aggregation: Retrieves links to news articles from a dedicated API.
- Content Scraping: Custom scraper extracts content from each linked website.
- LLM Processing: Uses a Large Language Model to create summaries, tags, and additional metadata.
- Data Storage: Processed information is stored in a database for easy access.
- Multiple Access Points: Users can access the summarized news via a React frontend or through an API.
Workflow
- API Integration: Fetch links to space-related news articles.
- Scraping: Navigate through each link to extract the full article content.
- Processing: Generate summaries and tags using an LLM.
- Storage: Save the processed content in a database.
- Access: Provide access to the stored information through a React frontend or an API.