View Issue Details

IDProjectCategoryView StatusLast Update
3322Composrcorepublic2020-03-21 01:14
ReporterChris Graham Assigned ToChris Graham  
PrioritynormalSeverityfeature 
Status resolvedResolutionfixed 
Summary3322: Cleanup broken URL APIs
DescriptionWe have:
 - test_url [for Comcode]
 - handle_has_checked_recently (uses url_title_cache table)
 - check_url_exists (uses urls_checked table)
 - check in broken URL API

Possibly these can be merged into a single API, using a single table.

It should offer good default functionality. It should be able to store HTTP responses in the table. It should be able to have "only cache if 200 status" functionality (i.e. cache on success, re-try on failure).
TagsRoadmap: v11
Attach Tags
Time estimation (hours)2
Sponsorship open

Sponsor

Date Added Member Amount Sponsored

Relationships

related to 3696 Not Assigned Google Webmaster Tools errors 

Activities

Chris Graham

2019-06-11 01:39

administrator   ~5963

Last edited: 2020-03-21 00:49

"Also having an admin_broken_urls module (v11) is highly questionable. Likely should just roll into Health Check, as a number of separate checks running off of a common API.

Should be different health checks for local URLs, remote URLs, and the other options currently handled in admin_broken_urls."

EDIT: No, because false-positives will happen.

Chris Graham

2020-01-26 23:08

administrator   ~6310

One thing to note is Cloudflare is sometimes putting out 'challenge pages', with a 403 status code and no way (other than screen-scraping) to know it's because it thinks you may be a bot. Which will be the case for broken URL checks.

So the broken URL checking needs to handle 403 (and probably 401) as not failures. Perhaps make the checks smarter than binary fails/works, with support for a few different response categories, and include this information in the check results.

Issue History

Date Modified Username Field Change
2017-07-23 21:58 Chris Graham New Issue
2017-09-03 22:47 Chris Graham Description Updated
2017-09-03 22:48 Chris Graham Description Updated
2019-06-11 01:39 Chris Graham Note Added: 0005963
2019-06-17 16:47 Chris Graham Relationship added related to 3696
2019-06-27 17:58 Chris Graham Tag Attached: Roadmap: v11
2020-01-26 23:08 Chris Graham Note Added: 0006310
2020-03-07 21:20 Chris Graham Assigned To => Chris Graham
2020-03-07 21:20 Chris Graham Status Not Assigned => Assigned
2020-03-21 00:49 Chris Graham Note Edited: 0005963
2020-03-21 01:14 Chris Graham Status Assigned => Resolved
2020-03-21 01:14 Chris Graham Resolution open => fixed