mirror of
https://github.com/Shubhamsaboo/awesome-llm-apps.git
synced 2026-09-21 05:10:55 +08:00
feat(devpulse_ai): add multi-agent signal intelligence reference implementation
This commit is contained in:
@@ -0,0 +1,150 @@
|
||||
## 🧠 DevPulseAI - Multi-Agent Signal Intelligence Pipeline
|
||||
|
||||
A reference implementation demonstrating a **multi-agent system** for aggregating, analyzing, and synthesizing technical signals from multiple developer-focused sources.
|
||||
|
||||
### Features
|
||||
|
||||
- **Multi-Source Signal Collection** - Aggregates data from GitHub, ArXiv, HackerNews, Medium, and HuggingFace
|
||||
- **LLM-Powered Analysis** - Four specialized agents working in concert
|
||||
- **Structured Intelligence Output** - Prioritized digest with actionable recommendations
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Signal Intelligence Pipeline │
|
||||
├─────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ │
|
||||
│ │ GitHub │ │ ArXiv │ │ HN │ │ Medium │ │ HF │ ← Data │
|
||||
│ └───┬────┘ └───┬────┘ └───┬────┘ └───┬────┘ └───┬────┘ │
|
||||
│ └──────────┴──────────┼──────────┴──────────┘ │
|
||||
│ │ │ │ │
|
||||
│ └─────────────┼─────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────┐ │
|
||||
│ │ Signal Collector│ ← Agent 1: Ingestion │
|
||||
│ └────────┬────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────┐ │
|
||||
│ │ Relevance Agent │ ← Agent 2: Scoring (0-100) │
|
||||
│ └────────┬────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────┐ │
|
||||
│ │ Risk Agent │ ← Agent 3: Security Assessment │
|
||||
│ └────────┬────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────┐ │
|
||||
│ │ Synthesis Agent │ ← Agent 4: Final Digest │
|
||||
│ └────────┬────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────┐ │
|
||||
│ │ Intelligence │ ← Prioritized Output │
|
||||
│ │ Digest │ │
|
||||
│ └─────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Agent Responsibilities
|
||||
|
||||
| Agent | Role | Output |
|
||||
|-------|------|--------|
|
||||
| **SignalCollectorAgent** | Aggregates & normalizes signals | Unified signal list |
|
||||
| **RelevanceAgent** | Scores developer relevance (0-100) | Score + reasoning |
|
||||
| **RiskAgent** | Identifies security/breaking changes | Risk level + concerns |
|
||||
| **SynthesisAgent** | Produces final intelligence digest | Prioritized recommendations |
|
||||
|
||||
### How to Get Started
|
||||
|
||||
1. Clone the repository
|
||||
|
||||
```bash
|
||||
git clone https://github.com/Shubhamsaboo/awesome-llm-apps.git
|
||||
cd advanced_ai_agents/multi_agent_apps/devpulse_ai
|
||||
```
|
||||
|
||||
1. Install dependencies
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
1. Set your OpenAI API key (optional for live mode)
|
||||
|
||||
```bash
|
||||
export OPENAI_API_KEY=your_api_key
|
||||
```
|
||||
|
||||
1. Run the verification script (no API key needed)
|
||||
|
||||
```bash
|
||||
python verify.py
|
||||
```
|
||||
|
||||
1. Run the full pipeline (requires API key for LLM agents)
|
||||
|
||||
```bash
|
||||
python main.py
|
||||
```
|
||||
|
||||
### Verification Script
|
||||
|
||||
The `verify.py` script tests the entire pipeline using **mock data only** - no network calls or API keys required:
|
||||
|
||||
```bash
|
||||
python verify.py
|
||||
```
|
||||
|
||||
Expected output:
|
||||
|
||||
```
|
||||
[OK] DevPulseAI reference pipeline executed successfully
|
||||
```
|
||||
|
||||
### Optional: n8n Automation
|
||||
|
||||
An n8n workflow is included for those who want to automate the pipeline:
|
||||
|
||||
- **Location**: `workflows/signal-intelligence-pipeline.json`
|
||||
- **Import**: n8n → Settings → Import from File
|
||||
- **Requires**: n8n instance + configured credentials
|
||||
|
||||
This is entirely optional - the Python implementation works standalone.
|
||||
|
||||
### Directory Structure
|
||||
|
||||
```
|
||||
devpulse_ai/
|
||||
├── adapters/
|
||||
│ ├── github.py # GitHub trending repos
|
||||
│ ├── arxiv.py # AI/ML research papers
|
||||
│ ├── hackernews.py # Tech news stories
|
||||
│ ├── medium.py # Tech blog RSS feeds
|
||||
│ └── huggingface.py # HuggingFace models
|
||||
├── agents/
|
||||
│ ├── __init__.py
|
||||
│ ├── signal_collector.py
|
||||
│ ├── relevance_agent.py
|
||||
│ ├── risk_agent.py
|
||||
│ └── synthesis_agent.py
|
||||
├── workflows/
|
||||
│ └── signal-intelligence-pipeline.json
|
||||
├── main.py # Full pipeline demo
|
||||
├── verify.py # Mock data verification
|
||||
├── requirements.txt
|
||||
└── README.md
|
||||
```
|
||||
|
||||
### How It Works
|
||||
|
||||
1. **Signal Collection**: Adapters fetch data from GitHub, ArXiv, HackerNews, Medium, and HuggingFace
|
||||
2. **Normalization**: SignalCollectorAgent unifies signals to a common schema
|
||||
3. **Relevance Scoring**: RelevanceAgent rates each signal 0-100 for developer relevance
|
||||
4. **Risk Assessment**: RiskAgent flags security issues and breaking changes
|
||||
5. **Synthesis**: SynthesisAgent produces a prioritized intelligence digest
|
||||
|
||||
### Built With
|
||||
|
||||
- [Agno](https://github.com/agno-agi/agno) - Multi-agent framework
|
||||
- [OpenAI GPT-4o-mini](https://openai.com/) - LLM backbone
|
||||
- [httpx](https://www.python-httpx.org/) - Async HTTP client
|
||||
BIN
Binary file not shown.
BIN
Binary file not shown.
BIN
Binary file not shown.
BIN
Binary file not shown.
BIN
Binary file not shown.
@@ -0,0 +1,86 @@
|
||||
"""
|
||||
ArXiv Adapter - Fetches recent AI/ML research papers.
|
||||
|
||||
This is a simplified, stateless adapter for the DevPulseAI reference implementation.
|
||||
ArXiv API is public and requires no authentication.
|
||||
"""
|
||||
|
||||
import httpx
|
||||
import xml.etree.ElementTree as ET
|
||||
from typing import List, Dict, Any
|
||||
|
||||
|
||||
def fetch_arxiv_papers(limit: int = 5) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Fetch recent AI/ML papers from ArXiv.
|
||||
|
||||
Args:
|
||||
limit: Maximum number of papers to return.
|
||||
|
||||
Returns:
|
||||
List of signal dictionaries with standardized schema.
|
||||
"""
|
||||
base_url = "https://export.arxiv.org/api/query"
|
||||
params = {
|
||||
"search_query": "cat:cs.AI OR cat:cs.LG",
|
||||
"start": 0,
|
||||
"max_results": limit,
|
||||
"sortBy": "submittedDate",
|
||||
"sortOrder": "descending"
|
||||
}
|
||||
|
||||
signals = []
|
||||
|
||||
try:
|
||||
response = httpx.get(base_url, params=params, timeout=15.0)
|
||||
response.raise_for_status()
|
||||
|
||||
# Parse Atom XML response
|
||||
root = ET.fromstring(response.content)
|
||||
ns = {"atom": "http://www.w3.org/2005/Atom"}
|
||||
|
||||
for entry in root.findall("atom:entry", ns):
|
||||
title_elem = entry.find("atom:title", ns)
|
||||
summary_elem = entry.find("atom:summary", ns)
|
||||
id_elem = entry.find("atom:id", ns)
|
||||
published_elem = entry.find("atom:published", ns)
|
||||
|
||||
title = title_elem.text.strip() if title_elem is not None else "Untitled"
|
||||
summary = summary_elem.text.strip() if summary_elem is not None else ""
|
||||
arxiv_id = id_elem.text.strip() if id_elem is not None else ""
|
||||
published = published_elem.text if published_elem is not None else ""
|
||||
|
||||
# Get PDF link
|
||||
pdf_link = arxiv_id
|
||||
link_elem = entry.find("atom:link[@title='pdf']", ns)
|
||||
if link_elem is not None:
|
||||
pdf_link = link_elem.attrib.get("href", arxiv_id)
|
||||
|
||||
signal = {
|
||||
"id": arxiv_id,
|
||||
"source": "arxiv",
|
||||
"title": title,
|
||||
"description": summary[:500] + "..." if len(summary) > 500 else summary,
|
||||
"url": arxiv_id,
|
||||
"metadata": {
|
||||
"pdf": pdf_link,
|
||||
"published": published
|
||||
}
|
||||
}
|
||||
signals.append(signal)
|
||||
|
||||
except httpx.HTTPError as e:
|
||||
print(f"[ArXiv Adapter] HTTP error: {e}")
|
||||
except ET.ParseError as e:
|
||||
print(f"[ArXiv Adapter] XML parse error: {e}")
|
||||
except Exception as e:
|
||||
print(f"[ArXiv Adapter] Error: {e}")
|
||||
|
||||
return signals
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Quick test
|
||||
results = fetch_arxiv_papers(limit=3)
|
||||
for r in results:
|
||||
print(f"- {r['title'][:60]}...")
|
||||
@@ -0,0 +1,65 @@
|
||||
"""
|
||||
GitHub Adapter - Fetches trending repositories from GitHub.
|
||||
|
||||
This is a simplified, stateless adapter for the DevPulseAI reference implementation.
|
||||
No authentication required for basic public API access.
|
||||
"""
|
||||
|
||||
import httpx
|
||||
from datetime import datetime, timedelta
|
||||
from typing import List, Dict, Any
|
||||
|
||||
|
||||
def fetch_github_trending(limit: int = 5) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Fetch trending GitHub repositories created in the last 24 hours.
|
||||
|
||||
Args:
|
||||
limit: Maximum number of repositories to return.
|
||||
|
||||
Returns:
|
||||
List of signal dictionaries with standardized schema.
|
||||
"""
|
||||
base_url = "https://api.github.com/search/repositories"
|
||||
date_query = (datetime.utcnow() - timedelta(days=1)).strftime("%Y-%m-%d")
|
||||
|
||||
params = {
|
||||
"q": f"created:>{date_query} sort:stars",
|
||||
"per_page": limit
|
||||
}
|
||||
|
||||
signals = []
|
||||
|
||||
try:
|
||||
response = httpx.get(base_url, params=params, timeout=10.0)
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
|
||||
for item in data.get("items", []):
|
||||
signal = {
|
||||
"id": str(item["id"]),
|
||||
"source": "github",
|
||||
"title": item["full_name"],
|
||||
"description": item.get("description") or "No description",
|
||||
"url": item["html_url"],
|
||||
"metadata": {
|
||||
"stars": item["stargazers_count"],
|
||||
"language": item.get("language"),
|
||||
"topics": item.get("topics", [])
|
||||
}
|
||||
}
|
||||
signals.append(signal)
|
||||
|
||||
except httpx.HTTPError as e:
|
||||
print(f"[GitHub Adapter] HTTP error: {e}")
|
||||
except Exception as e:
|
||||
print(f"[GitHub Adapter] Error: {e}")
|
||||
|
||||
return signals
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Quick test
|
||||
results = fetch_github_trending(limit=3)
|
||||
for r in results:
|
||||
print(f"- {r['title']}: {r['metadata']['stars']} stars")
|
||||
@@ -0,0 +1,72 @@
|
||||
"""
|
||||
HackerNews Adapter - Fetches top AI/ML stories from HackerNews.
|
||||
|
||||
This is a simplified, stateless adapter for the DevPulseAI reference implementation.
|
||||
Uses the Algolia HN API for better search capabilities.
|
||||
"""
|
||||
|
||||
import httpx
|
||||
from typing import List, Dict, Any
|
||||
|
||||
|
||||
def fetch_hackernews_stories(limit: int = 5) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Fetch recent AI/ML related stories from HackerNews.
|
||||
|
||||
Args:
|
||||
limit: Maximum number of stories to return.
|
||||
|
||||
Returns:
|
||||
List of signal dictionaries with standardized schema.
|
||||
"""
|
||||
base_url = "https://hn.algolia.com/api/v1/search_by_date"
|
||||
params = {
|
||||
"query": "AI OR LLM OR Machine Learning OR GPT",
|
||||
"tags": "story",
|
||||
"hitsPerPage": limit,
|
||||
"numericFilters": "points>5"
|
||||
}
|
||||
|
||||
signals = []
|
||||
|
||||
try:
|
||||
response = httpx.get(base_url, params=params, timeout=10.0)
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
|
||||
for hit in data.get("hits", []):
|
||||
# Skip stories without URLs (Ask HN, etc.)
|
||||
if not hit.get("url") and not hit.get("story_text"):
|
||||
continue
|
||||
|
||||
external_id = str(hit.get("objectID", ""))
|
||||
hn_url = f"https://news.ycombinator.com/item?id={external_id}"
|
||||
|
||||
signal = {
|
||||
"id": external_id,
|
||||
"source": "hackernews",
|
||||
"title": hit.get("title", "Untitled"),
|
||||
"description": hit.get("story_text", "")[:300] if hit.get("story_text") else "",
|
||||
"url": hit.get("url") or hn_url,
|
||||
"metadata": {
|
||||
"points": hit.get("points", 0),
|
||||
"comments": hit.get("num_comments", 0),
|
||||
"author": hit.get("author", "unknown"),
|
||||
"hn_url": hn_url
|
||||
}
|
||||
}
|
||||
signals.append(signal)
|
||||
|
||||
except httpx.HTTPError as e:
|
||||
print(f"[HackerNews Adapter] HTTP error: {e}")
|
||||
except Exception as e:
|
||||
print(f"[HackerNews Adapter] Error: {e}")
|
||||
|
||||
return signals
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Quick test
|
||||
results = fetch_hackernews_stories(limit=3)
|
||||
for r in results:
|
||||
print(f"- {r['title']}: {r['metadata']['points']} points")
|
||||
@@ -0,0 +1,80 @@
|
||||
"""
|
||||
HuggingFace Adapter - Fetches trending models from HuggingFace Hub.
|
||||
|
||||
This is a simplified, stateless adapter for the DevPulseAI reference implementation.
|
||||
Uses the public HuggingFace API (no authentication required for basic access).
|
||||
"""
|
||||
|
||||
import httpx
|
||||
from typing import List, Dict, Any
|
||||
|
||||
|
||||
def fetch_huggingface_models(limit: int = 5) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Fetch trending/popular models from HuggingFace Hub.
|
||||
|
||||
Args:
|
||||
limit: Maximum number of models to return.
|
||||
|
||||
Returns:
|
||||
List of signal dictionaries with standardized schema.
|
||||
"""
|
||||
base_url = "https://huggingface.co/api/models"
|
||||
params = {
|
||||
"sort": "likes",
|
||||
"direction": "-1",
|
||||
"limit": limit
|
||||
}
|
||||
|
||||
signals = []
|
||||
|
||||
try:
|
||||
response = httpx.get(base_url, params=params, timeout=10.0)
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
|
||||
for item in data:
|
||||
model_id = item.get("modelId", item.get("id", "unknown"))
|
||||
|
||||
# Build description from model metadata
|
||||
tags = item.get("tags", [])
|
||||
pipeline = item.get("pipeline_tag", "")
|
||||
description_parts = []
|
||||
|
||||
if pipeline:
|
||||
description_parts.append(f"Pipeline: {pipeline}")
|
||||
if tags:
|
||||
description_parts.append(f"Tags: {', '.join(tags[:5])}")
|
||||
|
||||
description_parts.append(f"Downloads: {item.get('downloads', 0):,}")
|
||||
description_parts.append(f"Likes: {item.get('likes', 0):,}")
|
||||
|
||||
signal = {
|
||||
"id": model_id,
|
||||
"source": "huggingface",
|
||||
"title": f"HF Model: {model_id}",
|
||||
"description": " | ".join(description_parts),
|
||||
"url": f"https://huggingface.co/{model_id}",
|
||||
"metadata": {
|
||||
"downloads": item.get("downloads", 0),
|
||||
"likes": item.get("likes", 0),
|
||||
"pipeline_tag": pipeline,
|
||||
"tags": tags[:10],
|
||||
"author": item.get("author", "")
|
||||
}
|
||||
}
|
||||
signals.append(signal)
|
||||
|
||||
except httpx.HTTPError as e:
|
||||
print(f"[HuggingFace Adapter] HTTP error: {e}")
|
||||
except Exception as e:
|
||||
print(f"[HuggingFace Adapter] Error: {e}")
|
||||
|
||||
return signals
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Quick test
|
||||
results = fetch_huggingface_models(limit=3)
|
||||
for r in results:
|
||||
print(f"- {r['title']}: {r['metadata']['likes']} likes")
|
||||
@@ -0,0 +1,70 @@
|
||||
"""
|
||||
Medium Adapter - Fetches tech blogs from Medium and other RSS feeds.
|
||||
|
||||
This is a simplified, stateless adapter for the DevPulseAI reference implementation.
|
||||
Uses feedparser to fetch from RSS/Atom feeds.
|
||||
"""
|
||||
|
||||
import feedparser
|
||||
from typing import List, Dict, Any
|
||||
|
||||
|
||||
# Tech blog feeds to monitor
|
||||
FEEDS = [
|
||||
"https://medium.com/feed/tag/artificial-intelligence",
|
||||
"https://medium.com/feed/tag/machine-learning",
|
||||
"https://medium.com/feed/@netflixtechblog",
|
||||
"https://engineering.fb.com/feed/",
|
||||
]
|
||||
|
||||
|
||||
def fetch_medium_blogs(limit: int = 5) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Fetch recent tech blogs from Medium and engineering blogs.
|
||||
|
||||
Args:
|
||||
limit: Maximum number of entries per feed.
|
||||
|
||||
Returns:
|
||||
List of signal dictionaries with standardized schema.
|
||||
"""
|
||||
signals = []
|
||||
|
||||
for feed_url in FEEDS:
|
||||
try:
|
||||
feed = feedparser.parse(feed_url)
|
||||
|
||||
for entry in feed.entries[:limit]:
|
||||
# Get summary or description
|
||||
summary = getattr(entry, "summary", "") or getattr(entry, "description", "")
|
||||
|
||||
# Clean HTML tags from summary (simple approach)
|
||||
if summary:
|
||||
import re
|
||||
summary = re.sub(r'<[^>]+>', '', summary)[:500]
|
||||
|
||||
signal = {
|
||||
"id": entry.get("id", entry.link),
|
||||
"source": "medium",
|
||||
"title": entry.title,
|
||||
"description": summary,
|
||||
"url": entry.link,
|
||||
"metadata": {
|
||||
"published": getattr(entry, "published", ""),
|
||||
"author": getattr(entry, "author", "Unknown"),
|
||||
"feed": feed_url
|
||||
}
|
||||
}
|
||||
signals.append(signal)
|
||||
|
||||
except Exception as e:
|
||||
print(f"[Medium Adapter] Error fetching {feed_url}: {e}")
|
||||
|
||||
return signals
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Quick test
|
||||
results = fetch_medium_blogs(limit=2)
|
||||
for r in results:
|
||||
print(f"- {r['title'][:60]}...")
|
||||
@@ -0,0 +1,21 @@
|
||||
"""
|
||||
DevPulseAI Agents Package
|
||||
|
||||
This package contains four specialized agents for the signal intelligence pipeline:
|
||||
- SignalCollectorAgent: Aggregates signals from multiple sources
|
||||
- RelevanceAgent: Scores signals based on developer relevance
|
||||
- RiskAgent: Assesses security risks and breaking changes
|
||||
- SynthesisAgent: Produces final intelligence digest
|
||||
"""
|
||||
|
||||
from .signal_collector import SignalCollectorAgent
|
||||
from .relevance_agent import RelevanceAgent
|
||||
from .risk_agent import RiskAgent
|
||||
from .synthesis_agent import SynthesisAgent
|
||||
|
||||
__all__ = [
|
||||
"SignalCollectorAgent",
|
||||
"RelevanceAgent",
|
||||
"RiskAgent",
|
||||
"SynthesisAgent"
|
||||
]
|
||||
BIN
Binary file not shown.
BIN
Binary file not shown.
BIN
Binary file not shown.
BIN
Binary file not shown.
BIN
Binary file not shown.
@@ -0,0 +1,121 @@
|
||||
"""
|
||||
Relevance Agent - Scores signals based on developer relevance.
|
||||
|
||||
This agent uses LLM reasoning to score each signal from 0-100
|
||||
based on its relevance to AI/ML developers and engineers.
|
||||
"""
|
||||
|
||||
from typing import Dict, Any, Optional
|
||||
from agno.agent import Agent
|
||||
from agno.models.openai import OpenAIChat
|
||||
|
||||
|
||||
class RelevanceAgent:
|
||||
"""
|
||||
Agent that scores signals based on relevance to developers.
|
||||
|
||||
Responsibilities:
|
||||
- Score signals 0-100 based on developer relevance
|
||||
- Provide reasoning for each score
|
||||
- Prioritize actionable, timely content
|
||||
"""
|
||||
|
||||
def __init__(self, model_id: str = "gpt-4o-mini"):
|
||||
"""
|
||||
Initialize the Relevance Agent.
|
||||
|
||||
Args:
|
||||
model_id: OpenAI model to use for scoring.
|
||||
"""
|
||||
self.model_id = model_id
|
||||
self.agent = Agent(
|
||||
name="Relevance Scorer",
|
||||
model=OpenAIChat(id=model_id),
|
||||
role="Scores technical signals based on developer relevance",
|
||||
instructions=[
|
||||
"Score each signal from 0-100 based on relevance.",
|
||||
"Consider: novelty, impact, actionability, and timeliness.",
|
||||
"Prioritize signals relevant to AI/ML engineers.",
|
||||
"Provide brief reasoning for each score."
|
||||
],
|
||||
markdown=True
|
||||
)
|
||||
|
||||
def score(self, signal: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""
|
||||
Score a signal for relevance.
|
||||
|
||||
Args:
|
||||
signal: Signal dictionary to score.
|
||||
|
||||
Returns:
|
||||
Dictionary with score and reasoning.
|
||||
"""
|
||||
prompt = f"""
|
||||
Rate the relevance of this signal for AI/ML developers.
|
||||
Score from 0-100 where:
|
||||
- 0-30: Low relevance (noise, off-topic)
|
||||
- 31-60: Moderate relevance (interesting but not urgent)
|
||||
- 61-80: High relevance (important for developers to know)
|
||||
- 81-100: Critical relevance (must-know, actionable)
|
||||
|
||||
Signal:
|
||||
- Source: {signal.get('source', 'unknown')}
|
||||
- Title: {signal.get('title', 'Untitled')}
|
||||
- Description: {signal.get('description', '')[:500]}
|
||||
|
||||
Respond with ONLY a JSON object:
|
||||
{{"score": <number>, "reasoning": "<one sentence>"}}
|
||||
"""
|
||||
|
||||
try:
|
||||
response = self.agent.run(prompt, stream=False)
|
||||
# Parse response - in real use, would parse JSON
|
||||
return self._parse_response(response.content, signal)
|
||||
except Exception as e:
|
||||
return self._fallback_score(signal, str(e))
|
||||
|
||||
def score_batch(self, signals: list) -> list:
|
||||
"""
|
||||
Score multiple signals.
|
||||
|
||||
Args:
|
||||
signals: List of signal dictionaries.
|
||||
|
||||
Returns:
|
||||
List of signals with scores added.
|
||||
"""
|
||||
scored = []
|
||||
for signal in signals:
|
||||
result = self.score(signal)
|
||||
signal_with_score = {**signal, "relevance": result}
|
||||
scored.append(signal_with_score)
|
||||
return scored
|
||||
|
||||
def _parse_response(self, content: str, signal: Dict) -> Dict[str, Any]:
|
||||
"""Parse LLM response into structured output."""
|
||||
import json
|
||||
try:
|
||||
# Try to extract JSON from response
|
||||
content = content.strip()
|
||||
if "```" in content:
|
||||
content = content.split("```")[1].replace("json", "").strip()
|
||||
return json.loads(content)
|
||||
except:
|
||||
return self._fallback_score(signal, "Parse error")
|
||||
|
||||
def _fallback_score(self, signal: Dict, error: str) -> Dict[str, Any]:
|
||||
"""Provide fallback score when LLM call fails."""
|
||||
# Simple heuristic based on metadata
|
||||
score = 50 # Default moderate score
|
||||
metadata = signal.get("metadata", {})
|
||||
|
||||
if metadata.get("stars", 0) > 100:
|
||||
score += 20
|
||||
if metadata.get("points", 0) > 50:
|
||||
score += 15
|
||||
|
||||
return {
|
||||
"score": min(score, 100),
|
||||
"reasoning": f"Heuristic score (LLM unavailable: {error})"
|
||||
}
|
||||
@@ -0,0 +1,133 @@
|
||||
"""
|
||||
Risk Agent - Assesses security risks and breaking changes.
|
||||
|
||||
This agent analyzes signals for potential risks including:
|
||||
- Security vulnerabilities
|
||||
- Breaking changes in dependencies
|
||||
- Deprecation notices
|
||||
"""
|
||||
|
||||
from typing import Dict, Any, List
|
||||
from agno.agent import Agent
|
||||
from agno.models.openai import OpenAIChat
|
||||
|
||||
|
||||
class RiskAgent:
|
||||
"""
|
||||
Agent that assesses risk levels in technical signals.
|
||||
|
||||
Responsibilities:
|
||||
- Identify security vulnerabilities
|
||||
- Flag breaking changes
|
||||
- Detect deprecation notices
|
||||
- Rate overall risk level
|
||||
"""
|
||||
|
||||
RISK_LEVELS = ["LOW", "MEDIUM", "HIGH", "CRITICAL"]
|
||||
|
||||
def __init__(self, model_id: str = "gpt-4o-mini"):
|
||||
"""
|
||||
Initialize the Risk Agent.
|
||||
|
||||
Args:
|
||||
model_id: OpenAI model to use for risk assessment.
|
||||
"""
|
||||
self.model_id = model_id
|
||||
self.agent = Agent(
|
||||
name="Risk Assessor",
|
||||
model=OpenAIChat(id=model_id),
|
||||
role="Assesses security and breaking change risks in technical signals",
|
||||
instructions=[
|
||||
"Analyze signals for security vulnerabilities.",
|
||||
"Identify breaking changes that may affect developers.",
|
||||
"Flag deprecation notices and migration requirements.",
|
||||
"Rate risk level: LOW, MEDIUM, HIGH, or CRITICAL."
|
||||
],
|
||||
markdown=True
|
||||
)
|
||||
|
||||
def assess(self, signal: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""
|
||||
Assess risk level of a signal.
|
||||
|
||||
Args:
|
||||
signal: Signal dictionary to assess.
|
||||
|
||||
Returns:
|
||||
Dictionary with risk assessment.
|
||||
"""
|
||||
prompt = f"""
|
||||
Analyze this technical signal for risks:
|
||||
|
||||
Signal:
|
||||
- Source: {signal.get('source', 'unknown')}
|
||||
- Title: {signal.get('title', 'Untitled')}
|
||||
- Description: {signal.get('description', '')[:500]}
|
||||
|
||||
Assess for:
|
||||
1. Security vulnerabilities
|
||||
2. Breaking changes
|
||||
3. Deprecations
|
||||
|
||||
Respond with ONLY a JSON object:
|
||||
{{"risk_level": "LOW|MEDIUM|HIGH|CRITICAL", "concerns": ["<list of concerns>"], "breaking_changes": true|false}}
|
||||
"""
|
||||
|
||||
try:
|
||||
response = self.agent.run(prompt, stream=False)
|
||||
return self._parse_response(response.content, signal)
|
||||
except Exception as e:
|
||||
return self._fallback_assessment(signal, str(e))
|
||||
|
||||
def assess_batch(self, signals: list) -> list:
|
||||
"""
|
||||
Assess multiple signals for risk.
|
||||
|
||||
Args:
|
||||
signals: List of signal dictionaries.
|
||||
|
||||
Returns:
|
||||
List of signals with risk assessments added.
|
||||
"""
|
||||
assessed = []
|
||||
for signal in signals:
|
||||
result = self.assess(signal)
|
||||
signal_with_risk = {**signal, "risk": result}
|
||||
assessed.append(signal_with_risk)
|
||||
return assessed
|
||||
|
||||
def _parse_response(self, content: str, signal: Dict) -> Dict[str, Any]:
|
||||
"""Parse LLM response into structured output."""
|
||||
import json
|
||||
try:
|
||||
content = content.strip()
|
||||
if "```" in content:
|
||||
content = content.split("```")[1].replace("json", "").strip()
|
||||
return json.loads(content)
|
||||
except:
|
||||
return self._fallback_assessment(signal, "Parse error")
|
||||
|
||||
def _fallback_assessment(self, signal: Dict, error: str) -> Dict[str, Any]:
|
||||
"""Provide fallback assessment when LLM call fails."""
|
||||
title = signal.get("title", "").lower()
|
||||
|
||||
# Simple keyword-based heuristics
|
||||
risk_level = "LOW"
|
||||
concerns = []
|
||||
|
||||
risk_keywords = {
|
||||
"HIGH": ["vulnerability", "exploit", "CVE", "critical", "breach"],
|
||||
"MEDIUM": ["breaking", "deprecated", "removed", "migration"],
|
||||
}
|
||||
|
||||
for level, keywords in risk_keywords.items():
|
||||
if any(kw.lower() in title for kw in keywords):
|
||||
risk_level = level
|
||||
concerns.append(f"Keyword match: {level}")
|
||||
break
|
||||
|
||||
return {
|
||||
"risk_level": risk_level,
|
||||
"concerns": concerns if concerns else [f"Heuristic (LLM unavailable: {error})"],
|
||||
"breaking_changes": "breaking" in title.lower()
|
||||
}
|
||||
@@ -0,0 +1,100 @@
|
||||
"""
|
||||
Signal Collector Agent - Aggregates signals from multiple data sources.
|
||||
|
||||
This agent is responsible for the ingestion phase of the pipeline.
|
||||
It collects signals from GitHub, ArXiv, and HackerNews, then normalizes
|
||||
them into a unified schema for downstream processing.
|
||||
"""
|
||||
|
||||
from typing import List, Dict, Any
|
||||
from agno.agent import Agent
|
||||
from agno.models.openai import OpenAIChat
|
||||
|
||||
|
||||
class SignalCollectorAgent:
|
||||
"""
|
||||
Agent that collects and normalizes signals from multiple sources.
|
||||
|
||||
Responsibilities:
|
||||
- Fetch data from configured adapters
|
||||
- Normalize signals to unified schema
|
||||
- Deduplicate and filter low-quality signals
|
||||
"""
|
||||
|
||||
def __init__(self, model_id: str = "gpt-4o-mini"):
|
||||
"""
|
||||
Initialize the Signal Collector Agent.
|
||||
|
||||
Args:
|
||||
model_id: OpenAI model to use for signal processing.
|
||||
"""
|
||||
self.model_id = model_id
|
||||
self.agent = Agent(
|
||||
name="Signal Collector",
|
||||
model=OpenAIChat(id=model_id),
|
||||
role="Collects and normalizes technical signals from multiple sources",
|
||||
instructions=[
|
||||
"You aggregate signals from GitHub, ArXiv, and HackerNews.",
|
||||
"Normalize all signals to a consistent format.",
|
||||
"Filter out low-quality or duplicate signals.",
|
||||
"Prioritize signals relevant to AI/ML developers."
|
||||
],
|
||||
markdown=True
|
||||
)
|
||||
|
||||
def collect(self, signals: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Process and normalize collected signals.
|
||||
|
||||
Args:
|
||||
signals: Raw signals from adapters.
|
||||
|
||||
Returns:
|
||||
List of normalized signal dictionaries.
|
||||
"""
|
||||
normalized = []
|
||||
seen_ids = set()
|
||||
|
||||
for signal in signals:
|
||||
# Deduplicate
|
||||
signal_id = f"{signal.get('source', 'unknown')}:{signal.get('id', '')}"
|
||||
if signal_id in seen_ids:
|
||||
continue
|
||||
seen_ids.add(signal_id)
|
||||
|
||||
# Ensure required fields
|
||||
normalized_signal = {
|
||||
"id": signal.get("id", ""),
|
||||
"source": signal.get("source", "unknown"),
|
||||
"title": signal.get("title", "Untitled"),
|
||||
"description": signal.get("description", ""),
|
||||
"url": signal.get("url", ""),
|
||||
"metadata": signal.get("metadata", {}),
|
||||
"collected_at": self._get_timestamp()
|
||||
}
|
||||
normalized.append(normalized_signal)
|
||||
|
||||
return normalized
|
||||
|
||||
def _get_timestamp(self) -> str:
|
||||
"""Get current UTC timestamp."""
|
||||
from datetime import datetime
|
||||
return datetime.utcnow().isoformat() + "Z"
|
||||
|
||||
def summarize_collection(self, signals: List[Dict[str, Any]]) -> str:
|
||||
"""
|
||||
Generate a summary of collected signals.
|
||||
|
||||
Args:
|
||||
signals: List of collected signals.
|
||||
|
||||
Returns:
|
||||
Summary string.
|
||||
"""
|
||||
sources = {}
|
||||
for s in signals:
|
||||
src = s.get("source", "unknown")
|
||||
sources[src] = sources.get(src, 0) + 1
|
||||
|
||||
summary_parts = [f"{count} from {src}" for src, count in sources.items()]
|
||||
return f"Collected {len(signals)} signals: {', '.join(summary_parts)}"
|
||||
@@ -0,0 +1,154 @@
|
||||
"""
|
||||
Synthesis Agent - Produces final intelligence digest.
|
||||
|
||||
This agent combines outputs from all previous agents to create
|
||||
a comprehensive, actionable intelligence summary for developers.
|
||||
"""
|
||||
|
||||
from typing import Dict, Any, List
|
||||
from agno.agent import Agent
|
||||
from agno.models.openai import OpenAIChat
|
||||
|
||||
|
||||
class SynthesisAgent:
|
||||
"""
|
||||
Agent that synthesizes all signal intelligence into a final digest.
|
||||
|
||||
Responsibilities:
|
||||
- Combine relevance and risk assessments
|
||||
- Prioritize signals by importance
|
||||
- Generate executive summary
|
||||
- Produce actionable recommendations
|
||||
"""
|
||||
|
||||
def __init__(self, model_id: str = "gpt-4o-mini"):
|
||||
"""
|
||||
Initialize the Synthesis Agent.
|
||||
|
||||
Args:
|
||||
model_id: OpenAI model to use for synthesis.
|
||||
"""
|
||||
self.model_id = model_id
|
||||
self.agent = Agent(
|
||||
name="Intelligence Synthesizer",
|
||||
model=OpenAIChat(id=model_id),
|
||||
role="Synthesizes technical signals into actionable intelligence digests",
|
||||
instructions=[
|
||||
"Combine relevance scores and risk assessments.",
|
||||
"Prioritize by: high relevance + critical risks first.",
|
||||
"Generate an executive summary.",
|
||||
"Provide actionable recommendations for developers."
|
||||
],
|
||||
markdown=True
|
||||
)
|
||||
|
||||
def synthesize(self, signals: List[Dict[str, Any]]) -> Dict[str, Any]:
|
||||
"""
|
||||
Synthesize signals into a final intelligence digest.
|
||||
|
||||
Args:
|
||||
signals: List of signals with relevance and risk data.
|
||||
|
||||
Returns:
|
||||
Complete intelligence digest.
|
||||
"""
|
||||
# Sort by priority (high relevance + high risk first)
|
||||
prioritized = self._prioritize_signals(signals)
|
||||
|
||||
# Group by category
|
||||
grouped = self._group_by_source(prioritized)
|
||||
|
||||
# Generate summary
|
||||
summary = self._generate_summary(prioritized)
|
||||
|
||||
return {
|
||||
"generated_at": self._get_timestamp(),
|
||||
"total_signals": len(signals),
|
||||
"executive_summary": summary,
|
||||
"priority_signals": prioritized[:5], # Top 5
|
||||
"signals_by_source": grouped,
|
||||
"recommendations": self._generate_recommendations(prioritized)
|
||||
}
|
||||
|
||||
def _prioritize_signals(self, signals: List[Dict]) -> List[Dict]:
|
||||
"""Sort signals by priority score."""
|
||||
def priority_score(signal):
|
||||
relevance = signal.get("relevance", {}).get("score", 50)
|
||||
risk = signal.get("risk", {})
|
||||
risk_multiplier = {
|
||||
"CRITICAL": 2.0,
|
||||
"HIGH": 1.5,
|
||||
"MEDIUM": 1.0,
|
||||
"LOW": 0.8
|
||||
}.get(risk.get("risk_level", "LOW"), 1.0)
|
||||
return relevance * risk_multiplier
|
||||
|
||||
return sorted(signals, key=priority_score, reverse=True)
|
||||
|
||||
def _group_by_source(self, signals: List[Dict]) -> Dict[str, List]:
|
||||
"""Group signals by their source."""
|
||||
grouped = {}
|
||||
for signal in signals:
|
||||
source = signal.get("source", "unknown")
|
||||
if source not in grouped:
|
||||
grouped[source] = []
|
||||
grouped[source].append(signal)
|
||||
return grouped
|
||||
|
||||
def _generate_summary(self, signals: List[Dict]) -> str:
|
||||
"""Generate executive summary."""
|
||||
if not signals:
|
||||
return "No signals to summarize."
|
||||
|
||||
high_priority = [s for s in signals
|
||||
if s.get("relevance", {}).get("score", 0) >= 70]
|
||||
critical_risks = [s for s in signals
|
||||
if s.get("risk", {}).get("risk_level") in ["HIGH", "CRITICAL"]]
|
||||
|
||||
parts = [f"Analyzed {len(signals)} signals."]
|
||||
|
||||
if high_priority:
|
||||
parts.append(f"{len(high_priority)} high-relevance items detected.")
|
||||
|
||||
if critical_risks:
|
||||
parts.append(f"⚠️ {len(critical_risks)} signals with elevated risk.")
|
||||
|
||||
if signals:
|
||||
top = signals[0]
|
||||
parts.append(f"Top signal: {top.get('title', 'Unknown')}")
|
||||
|
||||
return " ".join(parts)
|
||||
|
||||
def _generate_recommendations(self, signals: List[Dict]) -> List[str]:
|
||||
"""Generate actionable recommendations."""
|
||||
recommendations = []
|
||||
|
||||
critical_risks = [s for s in signals
|
||||
if s.get("risk", {}).get("risk_level") == "CRITICAL"]
|
||||
if critical_risks:
|
||||
recommendations.append(
|
||||
f"🚨 Review {len(critical_risks)} critical-risk signals immediately"
|
||||
)
|
||||
|
||||
high_relevance = [s for s in signals
|
||||
if s.get("relevance", {}).get("score", 0) >= 80]
|
||||
if high_relevance:
|
||||
recommendations.append(
|
||||
f"📌 Prioritize {len(high_relevance)} high-relevance items"
|
||||
)
|
||||
|
||||
github_signals = [s for s in signals if s.get("source") == "github"]
|
||||
if github_signals:
|
||||
recommendations.append(
|
||||
f"⭐ Explore {len(github_signals)} trending repositories"
|
||||
)
|
||||
|
||||
if not recommendations:
|
||||
recommendations.append("✅ No urgent actions required")
|
||||
|
||||
return recommendations
|
||||
|
||||
def _get_timestamp(self) -> str:
|
||||
"""Get current UTC timestamp."""
|
||||
from datetime import datetime
|
||||
return datetime.utcnow().isoformat() + "Z"
|
||||
@@ -0,0 +1,143 @@
|
||||
"""
|
||||
DevPulseAI - Multi-Agent Signal Intelligence Pipeline
|
||||
|
||||
This script demonstrates a complete multi-agent workflow for
|
||||
aggregating and analyzing technical signals from multiple sources.
|
||||
|
||||
Usage:
|
||||
python main.py
|
||||
|
||||
Requirements:
|
||||
- OpenAI API key set as OPENAI_API_KEY environment variable
|
||||
- Internet connection for fetching live data
|
||||
"""
|
||||
|
||||
import os
|
||||
from typing import List, Dict, Any
|
||||
|
||||
# Import adapters
|
||||
from adapters.github import fetch_github_trending
|
||||
from adapters.arxiv import fetch_arxiv_papers
|
||||
from adapters.hackernews import fetch_hackernews_stories
|
||||
from adapters.medium import fetch_medium_blogs
|
||||
from adapters.huggingface import fetch_huggingface_models
|
||||
|
||||
# Import agents
|
||||
from agents import (
|
||||
SignalCollectorAgent,
|
||||
RelevanceAgent,
|
||||
RiskAgent,
|
||||
SynthesisAgent
|
||||
)
|
||||
|
||||
|
||||
def collect_signals() -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Collect signals from all configured sources.
|
||||
|
||||
Returns:
|
||||
Combined list of signals from all adapters.
|
||||
"""
|
||||
print("\n📡 [1/4] Collecting Signals...")
|
||||
|
||||
signals = []
|
||||
|
||||
# Fetch from each source
|
||||
print(" → Fetching GitHub trending repos...")
|
||||
signals.extend(fetch_github_trending(limit=5))
|
||||
|
||||
print(" → Fetching ArXiv papers...")
|
||||
signals.extend(fetch_arxiv_papers(limit=5))
|
||||
|
||||
print(" → Fetching HackerNews stories...")
|
||||
signals.extend(fetch_hackernews_stories(limit=5))
|
||||
|
||||
print(" → Fetching Medium blogs...")
|
||||
signals.extend(fetch_medium_blogs(limit=3))
|
||||
|
||||
print(" → Fetching HuggingFace models...")
|
||||
signals.extend(fetch_huggingface_models(limit=5))
|
||||
|
||||
print(f" ✓ Collected {len(signals)} raw signals")
|
||||
return signals
|
||||
|
||||
|
||||
def run_pipeline():
|
||||
"""
|
||||
Execute the full signal intelligence pipeline.
|
||||
|
||||
Pipeline stages:
|
||||
1. Signal Collection - Aggregate from multiple sources
|
||||
2. Relevance Scoring - Rate signals 0-100
|
||||
3. Risk Assessment - Identify security/breaking changes
|
||||
4. Synthesis - Produce final intelligence digest
|
||||
"""
|
||||
print("=" * 60)
|
||||
print("🧠 DevPulseAI - Signal Intelligence Pipeline")
|
||||
print("=" * 60)
|
||||
|
||||
# Check for API key
|
||||
if not os.environ.get("OPENAI_API_KEY"):
|
||||
print("\n⚠️ Warning: OPENAI_API_KEY not set.")
|
||||
print(" LLM-based agents will use fallback heuristics.\n")
|
||||
|
||||
# Stage 1: Collection
|
||||
raw_signals = collect_signals()
|
||||
|
||||
# Initialize agents
|
||||
collector = SignalCollectorAgent()
|
||||
relevance = RelevanceAgent()
|
||||
risk = RiskAgent()
|
||||
synthesis = SynthesisAgent()
|
||||
|
||||
# Stage 2: Normalize
|
||||
print("\n🔄 [2/4] Normalizing Signals...")
|
||||
normalized = collector.collect(raw_signals)
|
||||
print(f" ✓ {collector.summarize_collection(normalized)}")
|
||||
|
||||
# Stage 3: Score for relevance
|
||||
print("\n📊 [3/4] Scoring Relevance...")
|
||||
scored = relevance.score_batch(normalized)
|
||||
high_relevance = sum(1 for s in scored
|
||||
if s.get("relevance", {}).get("score", 0) >= 70)
|
||||
print(f" ✓ {high_relevance}/{len(scored)} signals rated high-relevance")
|
||||
|
||||
# Stage 4: Assess risks
|
||||
print("\n⚠️ [4/4] Assessing Risks...")
|
||||
assessed = risk.assess_batch(scored)
|
||||
critical = sum(1 for s in assessed
|
||||
if s.get("risk", {}).get("risk_level") in ["HIGH", "CRITICAL"])
|
||||
print(f" ✓ {critical}/{len(assessed)} signals with elevated risk")
|
||||
|
||||
# Stage 5: Synthesize
|
||||
print("\n📋 Generating Intelligence Digest...")
|
||||
digest = synthesis.synthesize(assessed)
|
||||
|
||||
# Output results
|
||||
print("\n" + "=" * 60)
|
||||
print("📄 INTELLIGENCE DIGEST")
|
||||
print("=" * 60)
|
||||
print(f"\n🕐 Generated: {digest['generated_at']}")
|
||||
print(f"📦 Total Signals: {digest['total_signals']}")
|
||||
print(f"\n📝 Summary: {digest['executive_summary']}")
|
||||
|
||||
print("\n🎯 Top Priority Signals:")
|
||||
for i, signal in enumerate(digest.get("priority_signals", [])[:3], 1):
|
||||
score = signal.get("relevance", {}).get("score", "?")
|
||||
risk_level = signal.get("risk", {}).get("risk_level", "?")
|
||||
print(f" {i}. [{signal['source']}] {signal['title'][:50]}...")
|
||||
print(f" Relevance: {score} | Risk: {risk_level}")
|
||||
|
||||
print("\n💡 Recommendations:")
|
||||
for rec in digest.get("recommendations", []):
|
||||
print(f" • {rec}")
|
||||
|
||||
print("\n" + "=" * 60)
|
||||
print("✅ Pipeline completed successfully!")
|
||||
print("=" * 60)
|
||||
|
||||
return digest
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
run_pipeline()
|
||||
@@ -0,0 +1,4 @@
|
||||
agno
|
||||
httpx
|
||||
openai
|
||||
feedparser
|
||||
@@ -0,0 +1,207 @@
|
||||
"""
|
||||
DevPulseAI Verification Script
|
||||
|
||||
This script verifies the pipeline works correctly using MOCK DATA ONLY.
|
||||
No network calls or API keys are required.
|
||||
|
||||
Usage:
|
||||
python verify.py
|
||||
|
||||
Expected output:
|
||||
[OK] DevPulseAI reference pipeline executed successfully
|
||||
"""
|
||||
|
||||
from typing import List, Dict, Any
|
||||
|
||||
# Mock signal data for verification
|
||||
MOCK_SIGNALS = [
|
||||
{
|
||||
"id": "mock-gh-001",
|
||||
"source": "github",
|
||||
"title": "awesome-llm-apps",
|
||||
"description": "A curated collection of awesome LLM apps built with RAG and AI agents.",
|
||||
"url": "https://github.com/Shubhamsaboo/awesome-llm-apps",
|
||||
"metadata": {"stars": 5000, "language": "Python", "topics": ["llm", "ai"]}
|
||||
},
|
||||
{
|
||||
"id": "mock-arxiv-001",
|
||||
"source": "arxiv",
|
||||
"title": "Attention Is All You Need: Revisited",
|
||||
"description": "A comprehensive analysis of transformer architectures and their evolution over the past years.",
|
||||
"url": "https://arxiv.org/abs/2401.00001",
|
||||
"metadata": {"pdf": "https://arxiv.org/pdf/2401.00001", "published": "2024-01-15"}
|
||||
},
|
||||
{
|
||||
"id": "mock-hn-001",
|
||||
"source": "hackernews",
|
||||
"title": "GPT-5 Breaking Changes in API",
|
||||
"description": "OpenAI announces breaking changes to the Chat Completions API.",
|
||||
"url": "https://news.ycombinator.com/item?id=12345",
|
||||
"metadata": {"points": 500, "comments": 200, "author": "techwriter"}
|
||||
},
|
||||
{
|
||||
"id": "mock-medium-001",
|
||||
"source": "medium",
|
||||
"title": "Building Production RAG Systems",
|
||||
"description": "A deep dive into building scalable retrieval-augmented generation pipelines.",
|
||||
"url": "https://medium.com/@techblog/building-rag",
|
||||
"metadata": {"author": "TechBlog", "published": "2024-01-20"}
|
||||
},
|
||||
{
|
||||
"id": "mock-hf-001",
|
||||
"source": "huggingface",
|
||||
"title": "HF Model: meta-llama/Llama-3-8B",
|
||||
"description": "Pipeline: text-generation | Downloads: 1,000,000 | Likes: 5,000",
|
||||
"url": "https://huggingface.co/meta-llama/Llama-3-8B",
|
||||
"metadata": {"downloads": 1000000, "likes": 5000, "pipeline_tag": "text-generation"}
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
def verify_imports():
|
||||
"""Verify all modules can be imported."""
|
||||
print("[1/5] Verifying imports...")
|
||||
|
||||
try:
|
||||
from agents import (
|
||||
SignalCollectorAgent,
|
||||
RelevanceAgent,
|
||||
RiskAgent,
|
||||
SynthesisAgent
|
||||
)
|
||||
from adapters.github import fetch_github_trending
|
||||
from adapters.arxiv import fetch_arxiv_papers
|
||||
from adapters.hackernews import fetch_hackernews_stories
|
||||
from adapters.medium import fetch_medium_blogs
|
||||
from adapters.huggingface import fetch_huggingface_models
|
||||
print(" ✓ All modules imported successfully")
|
||||
return True
|
||||
except ImportError as e:
|
||||
print(f" ✗ Import error: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def verify_signal_collector():
|
||||
"""Verify SignalCollectorAgent works with mock data."""
|
||||
print("[2/5] Verifying Signal Collector...")
|
||||
|
||||
from agents import SignalCollectorAgent
|
||||
|
||||
collector = SignalCollectorAgent()
|
||||
normalized = collector.collect(MOCK_SIGNALS)
|
||||
|
||||
assert len(normalized) == len(MOCK_SIGNALS), "Signal count mismatch"
|
||||
assert all("collected_at" in s for s in normalized), "Missing timestamp"
|
||||
|
||||
summary = collector.summarize_collection(normalized)
|
||||
print(f" ✓ {summary}")
|
||||
return normalized
|
||||
|
||||
|
||||
def verify_relevance_agent(signals: List[Dict]):
|
||||
"""Verify RelevanceAgent works with mock data."""
|
||||
print("[3/5] Verifying Relevance Agent...")
|
||||
|
||||
from agents import RelevanceAgent
|
||||
|
||||
# Use fallback mode (no API key needed)
|
||||
agent = RelevanceAgent()
|
||||
|
||||
scored = []
|
||||
for signal in signals:
|
||||
# Use fallback scoring directly
|
||||
result = agent._fallback_score(signal, "Mock mode")
|
||||
signal_with_score = {**signal, "relevance": result}
|
||||
scored.append(signal_with_score)
|
||||
|
||||
assert all("relevance" in s for s in scored), "Missing relevance scores"
|
||||
print(f" ✓ Scored {len(scored)} signals")
|
||||
return scored
|
||||
|
||||
|
||||
def verify_risk_agent(signals: List[Dict]):
|
||||
"""Verify RiskAgent works with mock data."""
|
||||
print("[4/5] Verifying Risk Agent...")
|
||||
|
||||
from agents import RiskAgent
|
||||
|
||||
agent = RiskAgent()
|
||||
|
||||
assessed = []
|
||||
for signal in signals:
|
||||
# Use fallback assessment directly
|
||||
result = agent._fallback_assessment(signal, "Mock mode")
|
||||
signal_with_risk = {**signal, "risk": result}
|
||||
assessed.append(signal_with_risk)
|
||||
|
||||
assert all("risk" in s for s in assessed), "Missing risk assessments"
|
||||
|
||||
# Check that breaking change detection works
|
||||
breaking = [s for s in assessed if s.get("risk", {}).get("breaking_changes")]
|
||||
print(f" ✓ Assessed {len(assessed)} signals ({len(breaking)} with breaking changes)")
|
||||
return assessed
|
||||
|
||||
|
||||
def verify_synthesis_agent(signals: List[Dict]):
|
||||
"""Verify SynthesisAgent produces valid digest."""
|
||||
print("[5/5] Verifying Synthesis Agent...")
|
||||
|
||||
from agents import SynthesisAgent
|
||||
|
||||
agent = SynthesisAgent()
|
||||
digest = agent.synthesize(signals)
|
||||
|
||||
assert "generated_at" in digest, "Missing timestamp"
|
||||
assert "executive_summary" in digest, "Missing summary"
|
||||
assert "recommendations" in digest, "Missing recommendations"
|
||||
assert digest["total_signals"] == len(signals), "Signal count mismatch"
|
||||
|
||||
print(f" ✓ Generated digest with {len(digest['recommendations'])} recommendations")
|
||||
return digest
|
||||
|
||||
|
||||
def run_verification():
|
||||
"""Run complete verification suite."""
|
||||
print("=" * 60)
|
||||
print("🔍 DevPulseAI Verification Suite")
|
||||
print("=" * 60)
|
||||
print("\nUsing MOCK DATA - No network calls or API keys required.\n")
|
||||
|
||||
try:
|
||||
# Step 1: Import verification
|
||||
if not verify_imports():
|
||||
raise AssertionError("Import verification failed")
|
||||
|
||||
# Step 2: Signal collection
|
||||
normalized = verify_signal_collector()
|
||||
|
||||
# Step 3: Relevance scoring
|
||||
scored = verify_relevance_agent(normalized)
|
||||
|
||||
# Step 4: Risk assessment
|
||||
assessed = verify_risk_agent(scored)
|
||||
|
||||
# Step 5: Synthesis
|
||||
digest = verify_synthesis_agent(assessed)
|
||||
|
||||
# Final summary
|
||||
print("\n" + "=" * 60)
|
||||
print("📊 Verification Summary")
|
||||
print("=" * 60)
|
||||
print(f" • Signals processed: {digest['total_signals']}")
|
||||
print(f" • Summary: {digest['executive_summary']}")
|
||||
print(f" • Recommendations: {len(digest['recommendations'])}")
|
||||
|
||||
print("\n" + "=" * 60)
|
||||
print("[OK] DevPulseAI reference pipeline executed successfully")
|
||||
print("=" * 60)
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
print(f"\n[FAIL] Verification failed: {e}")
|
||||
return False
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
success = run_verification()
|
||||
exit(0 if success else 1)
|
||||
+254
@@ -0,0 +1,254 @@
|
||||
{
|
||||
"name": "Signal Intelligence Ingestion Pipeline",
|
||||
"description": "Optional n8n workflow for automating the DevPulseAI signal intelligence pipeline. Import into n8n to schedule daily digest generation.",
|
||||
"nodes": [
|
||||
{
|
||||
"parameters": {},
|
||||
"id": "trigger-cron",
|
||||
"name": "Daily Trigger",
|
||||
"type": "n8n-nodes-base.scheduleTrigger",
|
||||
"typeVersion": 1.1,
|
||||
"position": [
|
||||
0,
|
||||
300
|
||||
],
|
||||
"notes": "Triggers the pipeline daily at configured time"
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"httpMethod": "POST",
|
||||
"path": "trigger-pulse",
|
||||
"responseMode": "responseNode"
|
||||
},
|
||||
"id": "webhook-trigger",
|
||||
"name": "Webhook Trigger",
|
||||
"type": "n8n-nodes-base.webhook",
|
||||
"typeVersion": 2,
|
||||
"position": [
|
||||
0,
|
||||
500
|
||||
],
|
||||
"webhookId": "devpulse-trigger",
|
||||
"notes": "Manual trigger via POST request"
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"url": "https://api.github.com/search/repositories?q=stars:>1000&sort=stars&order=desc&per_page=10",
|
||||
"options": {
|
||||
"response": {
|
||||
"response": {
|
||||
"responseFormat": "json"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"id": "github-adapter",
|
||||
"name": "GitHub Trending Repos",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
400,
|
||||
100
|
||||
],
|
||||
"notes": "Fetches trending GitHub repositories"
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"url": "http://export.arxiv.org/api/query?search_query=cat:cs.AI+OR+cat:cs.LG&sortBy=submittedDate&sortOrder=descending&max_results=10"
|
||||
},
|
||||
"id": "arxiv-adapter",
|
||||
"name": "ArXiv Papers",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
400,
|
||||
400
|
||||
],
|
||||
"notes": "Fetches latest AI/ML research papers"
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"url": "https://hn.algolia.com/api/v1/search_by_date?query=AI&tags=story&hitsPerPage=10"
|
||||
},
|
||||
"id": "hackernews-adapter",
|
||||
"name": "HackerNews Top Stories",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [
|
||||
400,
|
||||
550
|
||||
],
|
||||
"notes": "Fetches top HackerNews stories"
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"mode": "multiplex"
|
||||
},
|
||||
"id": "merge-signals",
|
||||
"name": "Aggregate Signals",
|
||||
"type": "n8n-nodes-base.merge",
|
||||
"typeVersion": 3,
|
||||
"position": [
|
||||
650,
|
||||
400
|
||||
],
|
||||
"notes": "Combines all signal sources into unified stream"
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"resource": "chat",
|
||||
"model": "gpt-4o-mini",
|
||||
"prompt": {
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"content": "You are a Relevance Scoring Agent. Score the following content from 0-100 based on its relevance to AI/ML developers. Return ONLY a JSON object: {\"score\": <number>, \"reason\": \"<1 sentence>\"}"
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Title: {{ $json.title }}\nDescription: {{ $json.description }}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"options": {
|
||||
"temperature": 0.1,
|
||||
"maxOutputTokens": 100
|
||||
}
|
||||
},
|
||||
"id": "relevance-agent",
|
||||
"name": "Relevance Agent",
|
||||
"type": "@n8n/n8n-nodes-langchain.lmChatOpenAi",
|
||||
"typeVersion": 1,
|
||||
"position": [
|
||||
1100,
|
||||
400
|
||||
],
|
||||
"notes": "Scores content 0-100 based on developer relevance"
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"resource": "chat",
|
||||
"model": "gpt-4o-mini",
|
||||
"prompt": {
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"content": "You are a Risk Assessment Agent. Analyze for: breaking changes, security vulnerabilities, or deprecations. Return ONLY a JSON object: {\"risk_level\": \"HIGH|MEDIUM|LOW\", \"concerns\": [\"<list>\"]}"
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Title: {{ $json.title }}\nDescription: {{ $json.description }}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"options": {
|
||||
"temperature": 0.1,
|
||||
"maxOutputTokens": 150
|
||||
}
|
||||
},
|
||||
"id": "risk-agent",
|
||||
"name": "Risk Agent",
|
||||
"type": "@n8n/n8n-nodes-langchain.lmChatOpenAi",
|
||||
"typeVersion": 1,
|
||||
"position": [
|
||||
1100,
|
||||
600
|
||||
],
|
||||
"notes": "Flags breaking changes and security vulnerabilities"
|
||||
}
|
||||
],
|
||||
"connections": {
|
||||
"Daily Trigger": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "GitHub Trending Repos",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Webhook Trigger": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "GitHub Trending Repos",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"GitHub Trending Repos": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Aggregate Signals",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"ArXiv Papers": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Aggregate Signals",
|
||||
"type": "main",
|
||||
"index": 1
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"HackerNews Top Stories": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Aggregate Signals",
|
||||
"type": "main",
|
||||
"index": 2
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Aggregate Signals": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Relevance Agent",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
},
|
||||
{
|
||||
"node": "Risk Agent",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
},
|
||||
"settings": {
|
||||
"executionOrder": "v1"
|
||||
},
|
||||
"tags": [
|
||||
{
|
||||
"name": "signal-intelligence"
|
||||
},
|
||||
{
|
||||
"name": "automation"
|
||||
},
|
||||
{
|
||||
"name": "developer-tools"
|
||||
},
|
||||
{
|
||||
"name": "ai-agents"
|
||||
}
|
||||
],
|
||||
"meta": {
|
||||
"templateCredsSetupCompleted": false,
|
||||
"instanceId": "devpulse-reference"
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user