Debug: Campaign Processing Steps

Field	Value
ID	1
Title	Test Campaign 1
Keywords	N/A
Platforms	N/A
Status	active
Date From	2026-05-01
Date To	2026-05-31
Client Keywords	N/A
Monitor Links	N/A
Relevancy Threshold	0.70

Step 2: Collect Data from Platforms

API Endpoint: api/collect_data_step2.php (POST)

Database Tables: twitter_raw, youtube_raw, instagram_raw, news_raw, blogs_raw, facebook_raw (INSERT, SELECT COUNT)

Database Table: background_jobs (SELECT)

ℹ️ What this means: Background jobs track long-running scraping processes. If no jobs are found, it means either:
• Scraping completed and jobs were cleaned up
• Scraping happened directly without creating job records (this is normal)
• No background scraping has been started yet
✅ Check Step 3 (Data Collection Status) to see if data was actually collected - that's what matters!

No background jobs found for this campaign

This is normal! Background jobs are optional tracking records. The important thing is whether data was collected - check Step 3 below.

ℹ️ This collects 100 results from each selected platform. This may take 10-20 minutes.

Data Collection Status by Platform

ℹ️ Limits: Loaded from platform_limits table (managed via Settings > Platform Limit)

Platform	Table Name	Records Count	Limit	Status
Twitter	`twitter_raw`	0	`1000`	⚠ No Data
Youtube	`youtube_raw`	0	`100`	⚠ No Data
Instagram	`instagram_raw`	5	`100`	✓ Data Collected
Facebook	`facebook_raw`	3	`100`	✓ Data Collected
News	`news_raw`	0	`1000`	⚠ No Data
Blogs	`blogs_raw`	0	`1000`	⚠ No Data
Total Records	8	-	✓ Data Available	-

Step 3: AI Relevancy Analysis

API Endpoint: api/check_raw_mentions.php (POST)

OpenAI key: loaded from api_keys_db.api_keys (then OPENAI_API_KEY env, then config). Model: gpt-4o-mini.

Database Tables: twitter_raw, youtube_raw, instagram_raw, news_raw, blogs_raw, facebook_raw (SELECT), ai_relevancy_results (INSERT)

ℹ️ Run sends every raw row to the API with force_reprocess (full pass, re-scores by AI). Re-Run does the same. Batches of 50; large campaigns can take 15+ minutes.

AI Relevancy Results

Database Table: ai_jobs (SELECT)

No AI jobs found for this campaign. Click the button above to create AI jobs from raw data.

Step 4: Complete Processing & Save Results

API Endpoint: api/sync_campaign_articles.php (POST)

Database Table: campaign_articles (INSERT/UPDATE, SELECT COUNT, SELECT)

ℹ️ This finalizes all collected data and syncs relevant items to campaign_articles.

Campaign Articles (Final Processed Results)

Total Campaign Articles: 2

Breakdown by Platform:

Platform	Articles Count
instagram	1
twitter	1

Sample Articles (Latest 5):

ID	Platform	Platform Post ID	Title/Content	Relevancy Score	Relevancy Label	Sentiment	Sentiment Score	Raw Item ID	Raw Table	Created At
49410	twitter	N/A	Breaking: Our new feature is now live! 🚀 This is g...	0.8800	N/A	positive	0.7800	N/A	N/A	2026-04-22 13:29:23
49409	instagram	N/A	Check out our latest product launch! Amazing featu...	0.9000	N/A	positive	0.8500	N/A	N/A	2026-04-22 13:29:23

Additional: Background Jobs Status

Database Table: background_jobs (SELECT)

ℹ️ Background jobs track long-running scraping processes. This is optional tracking.

No background jobs found for this campaign

Additional: AI Processing Jobs Status

Database Table: ai_jobs (SELECT)

No AI jobs found for this campaign

📋 Complete Summary Report

📊 Processing Status Summary

Metric	Value
Campaign ID	1
Campaign Status	active
Relevancy Threshold	0.70
Total Raw Records Collected	8
AI Relevancy Results	0
Campaign Articles (Final)	2
Background Jobs	0
AI Jobs	0

🌐 External APIs Used

API Service	Provider	Usage
`apidojo~tweet-scraper`	Apify	Twitter data collection (Step 3)
`streamers~youtube-scraper`	Apify	YouTube data collection (Step 3)
`apify~instagram-post-scraper`	Apify	Instagram data collection (Step 3)
ScrapingDog API	ScrapingDog	News & Blogs data collection (Step 3)
OpenAI API	OpenAI	AI relevancy analysis (Step 5)
Gemini API	Google	AI relevancy analysis (Step 5, alternative)

📊 Database Tables Used

Step	API Endpoint	Table Name	Operation
Step 1	`api/start_background_scraping.php`	`campaigns_impact` (relevancy_threshold)	SELECT
Step 2	`api/collect_data_step2.php`	`twitter_raw`, `youtube_raw`, `instagram_raw`, `news_raw`, `blogs_raw`, `facebook_raw`	INSERT, SELECT COUNT
Step 3	`api/check_raw_mentions.php`	`twitter_raw`, `youtube_raw`, `instagram_raw`, `news_raw`, `blogs_raw`, `facebook_raw`, `ai_relevancy_results` (platform_post_id, content_hash, processing_state, raw_item_id, raw_table_name)	SELECT, INSERT
Step 4	`api/sync_campaign_articles.php`	`campaign_articles` (platform_post_id, matched_terms, ai_reasoning, relevancy_label, sentiment_score, raw_item_id, raw_table_name)	INSERT/UPDATE, SELECT COUNT, SELECT

🔄 Processing Flow Summary

Step	Description	API Endpoint	Duration
Step 1	Initialize scraping process	`api/start_background_scraping.php`	10-20 seconds
Step 2	Collect data from platforms (100 results per platform)	`api/collect_data_step2.php`	10-20 minutes
Step 3	AI relevancy analysis (50 mentions per batch)	`api/check_raw_mentions.php`	5-7 minutes
Step 4	Complete processing & save results	`api/sync_campaign_articles.php`	1-2 minutes