Debug: Campaign Processing Steps

Campaign ID: 269

🔗 Direct Link

Step 1: Initialize Scraping Process

API Endpoint: api/start_background_scraping.php (POST)

Database Table: campaigns_impact (SELECT)

FieldValue
ID269
TitleCoke
Keywords"Coke"
Platformstwitter
Statusactive
Date From2026-06-08
Date To2026-06-11
Client Keywords
Monitor Links
Relevancy Threshold0.70

ℹ️ This prepares the campaign for data collection. This step must be completed before Step 2.

Step 2: Collect Data from Platforms

API Endpoint: api/collect_data_step2.php (POST)

Database Tables: twitter_raw, youtube_raw, instagram_raw, news_raw, blogs_raw, facebook_raw (INSERT, SELECT COUNT)

Database Table: background_jobs (SELECT)

ℹ️ What this means: Background jobs track long-running scraping processes. If no jobs are found, it means either:
• Scraping completed and jobs were cleaned up
• Scraping happened directly without creating job records (this is normal)
• No background scraping has been started yet
Check Step 3 (Data Collection Status) to see if data was actually collected - that's what matters!

No background jobs found for this campaign

This is normal! Background jobs are optional tracking records. The important thing is whether data was collected - check Step 3 below.

ℹ️ This collects 100 results from each selected platform. This may take 10-20 minutes.

Data Collection Status by Platform

ℹ️ Limits: Loaded from platform_limits table (managed via Settings > Platform Limit)

PlatformTable NameRecords CountLimitStatusAction
Twittertwitter_raw1001000✓ Data Collected
Youtubeyoutube_raw0100⚠ No Data
Instagraminstagram_raw0100⚠ No Data
Facebookfacebook_raw0100⚠ No Data
Newsnews_raw01000⚠ No Data
Blogsblogs_raw01000⚠ No Data
Total Records100-✓ Data Available-

Step 3: AI Relevancy Analysis

API Endpoint: api/check_raw_mentions.php (POST)

OpenAI key: loaded from api_keys_db.api_keys (then OPENAI_API_KEY env, then config). Model: gpt-4o-mini.

Database Tables: twitter_raw, youtube_raw, instagram_raw, news_raw, blogs_raw, facebook_raw (SELECT), ai_relevancy_results (INSERT)

ℹ️ Run sends every raw row to the API with force_reprocess (full pass, re-scores by AI). Re-Run does the same. Batches of 50; large campaigns can take 15+ minutes.

AI Relevancy Results

Database Table: ai_jobs (SELECT)

No AI jobs found for this campaign. Click the button above to create AI jobs from raw data.

Total AI Relevancy Results: 100

Breakdown by Platform:

PlatformResults Count
twitter100

Breakdown by Relevancy:

Relevancy LabelCount

Breakdown by Sentiment:

SentimentCount
positive88
neutral9
negative3

Breakdown by Processing State:

Processing StateCount
analyzed100

Sample Results (Latest 5):

IDPlatformPlatform Post IDRelevancy ScoreRelevancy LabelSentimentProcessing StateRaw Item IDRaw TableCreated At
104761twitter2380.9000N/Apositiveanalyzed238twitter_x_api_raw2026-06-11 10:18:10
104773twitter2500.9000N/Apositiveanalyzed250twitter_x_api_raw2026-06-11 10:18:10
104771twitter2480.9000N/Apositiveanalyzed248twitter_x_api_raw2026-06-11 10:18:10
104739twitter2160.9000N/Apositiveanalyzed216twitter_x_api_raw2026-06-11 10:18:10
104776twitter2530.9000N/Apositiveanalyzed253twitter_x_api_raw2026-06-11 10:18:10

Step 4: Complete Processing & Save Results

API Endpoint: api/sync_campaign_articles.php (POST)

Database Table: campaign_articles (INSERT/UPDATE, SELECT COUNT, SELECT)

ℹ️ This finalizes all collected data and syncs relevant items to campaign_articles.

Campaign Articles (Final Processed Results)

Total Campaign Articles: 93

ℹ️ This will sync relevant items (relevancy score ≥ 70% - campaign threshold: 0.7) from ai_relevancy_results to campaign_articles. Note: 93 relevant items found, 0 can be synced (others may already exist in campaign_articles).

Breakdown by Platform:

PlatformArticles Count
twitter93

Sample Articles (Latest 5):

IDPlatformPlatform Post IDTitle/ContentRelevancy ScoreRelevancy LabelSentimentSentiment ScoreRaw Item IDRaw TableCreated At
51717twitter208#コカコーラ勝敗予想対決 に参戦中⚽第1戦は #ガチ勢の私はメキシコが勝利 と予想! @CocaCo...0.9000very_highpositive0.8000208twitter_x_api_raw2026-06-11 10:18:10
51718twitter210#コカコーラ勝敗予想対決 に参戦中⚽第2戦は #ガチ勢の私はブラジルが勝利 と予想! @CocaCo...0.9000very_highpositive0.8000210twitter_x_api_raw2026-06-11 10:18:10
51719twitter211#コカコーラ勝敗予想対決 に参戦中⚽第1戦は #エンジョイ勢の私はメキシコが勝利 と予想! @Coc...0.9000very_highpositive0.8000211twitter_x_api_raw2026-06-11 10:18:10
51720twitter212#コカコーラ勝敗予想対決 に参戦中⚽第2戦は #エンジョイ勢の私はブラジルが勝利 と予想! @Coc...0.9000very_highpositive0.8000212twitter_x_api_raw2026-06-11 10:18:10
51721twitter213#コカコーラ勝敗予想対決 に参戦中⚽第2戦は #エンジョイ勢の私はモロッコが勝利 と予想! @Coc...0.9000very_highpositive0.8000213twitter_x_api_raw2026-06-11 10:18:10

Campaign Articles Breakdown by Relevancy Label:

Relevancy LabelCount
high30
very_high63

Additional: Background Jobs Status

Database Table: background_jobs (SELECT)

ℹ️ Background jobs track long-running scraping processes. This is optional tracking.

No background jobs found for this campaign

Additional: AI Processing Jobs Status

Database Table: ai_jobs (SELECT)

No AI jobs found for this campaign

📋 Complete Summary Report

📊 Processing Status Summary

MetricValue
Campaign ID269
Campaign Statusactive
Relevancy Threshold0.70
Total Raw Records Collected100
AI Relevancy Results100
Campaign Articles (Final)93
Background Jobs0
AI Jobs0

🌐 External APIs Used

API ServiceProviderUsage
apidojo~tweet-scraperApifyTwitter data collection (Step 3)
streamers~youtube-scraperApifyYouTube data collection (Step 3)
apify~instagram-post-scraperApifyInstagram data collection (Step 3)
ScrapingDog APIScrapingDogNews & Blogs data collection (Step 3)
OpenAI APIOpenAIAI relevancy analysis (Step 5)
Gemini APIGoogleAI relevancy analysis (Step 5, alternative)

📊 Database Tables Used

StepAPI EndpointTable NameOperation
Step 1api/start_background_scraping.phpcampaigns_impact (relevancy_threshold)SELECT
Step 2api/collect_data_step2.phptwitter_raw, youtube_raw, instagram_raw, news_raw, blogs_raw, facebook_rawINSERT, SELECT COUNT
Step 3api/check_raw_mentions.phptwitter_raw, youtube_raw, instagram_raw, news_raw, blogs_raw, facebook_raw, ai_relevancy_results (platform_post_id, content_hash, processing_state, raw_item_id, raw_table_name)SELECT, INSERT
Step 4api/sync_campaign_articles.phpcampaign_articles (platform_post_id, matched_terms, ai_reasoning, relevancy_label, sentiment_score, raw_item_id, raw_table_name)INSERT/UPDATE, SELECT COUNT, SELECT

🔄 Processing Flow Summary

StepDescriptionAPI EndpointDuration
Step 1Initialize scraping processapi/start_background_scraping.php10-20 seconds
Step 2Collect data from platforms (100 results per platform)api/collect_data_step2.php10-20 minutes
Step 3AI relevancy analysis (50 mentions per batch)api/check_raw_mentions.php5-7 minutes
Step 4Complete processing & save resultsapi/sync_campaign_articles.php1-2 minutes

✓ Complete!

All steps debugged successfully!

Campaign ID: 269 | Raw Records: 100 | AI Results: 100 | Articles: 93