View Issue Details

IDProjectCategoryView StatusLast Update
3032Composrcorepublic2019-10-19 02:02
ReporterChris Graham Assigned ToChris Graham  
PrioritynormalSeverityfeature 
Status resolvedResolutionfixed 
Summary3032: Bulletproof spreadsheet importing
DescriptionYou can currently import CSV files, and it also supports text files with ";" as a delimiter (we don't advertise this).

This is pretty techie for some users.

We should ideally be able to handle any of these scenarios, using a clean API:
1) User accidentally uploads CSV with tab delimiter (TSV) not proper CSV
2) User accidentally uploads CSV with semicolon delimiter not proper CSV
3) User uploads a spreadsheet with no header row in a situation where we don't strictly need one (newsletter raw email list import comes to mind)
4) User uploads a CSV with old-Mac-style line endings (this should not happen in 2016 but some ageing codebases export bizarre stuff; the PHP enable auto_detect_line_endings setting may help)
5) User uploads a spreadsheet with a different character set, esp UTF-16 or ANSI (to solve this we'll probably need to have a character-set selector box and also support BOM markers)
7) Use uploads a very large files that won't fit in memory in one go
Additional InformationCode files that load CSV files at the moment...

adminzone/pages/modules/admin_newsletter.php
sources/hooks/systems/tasks/import_member_csv.php
sources/hooks/systems/tasks/import_newsletter_subscribers.php
sources/hooks/systems/tasks/import_catalogue.php
sources/newsletter.php
pages/modules/recommend.php

We'd want all this to be carefully unit tested.
TagsRoadmap: v11, Type: Cross-cutting feature , Type: Usability (major)
Attach Tags
Time estimation (hours)6
Sponsorship open

Sponsor

Date Added Member Amount Sponsored

Relationships

related to 1908 ResolvedChris Graham Support importing of spreadsheets other than CSV 
related to 3467 ResolvedChris Graham Support Unicode BOM markers 

Activities

There are no notes attached to this issue.

Issue History

Date Modified Username Field Change
2017-01-19 22:47 Chris Graham New Issue
2017-01-19 22:51 Chris Graham Tag Attached: Type: Usability (major)
2017-01-26 12:51 Chris Graham Description Updated
2017-01-26 12:51 Chris Graham Additional Information Updated
2017-05-01 16:05 Chris Graham Tag Attached: Type: Cross-cutting feature
2017-05-01 17:09 Chris Graham Relationship added related to 1908
2018-02-07 20:44 Chris Graham Relationship added related to 3467
2018-02-07 20:45 Chris Graham Description Updated
2018-02-13 01:23 Chris Graham Time estimation (hours) 12 => 6
2018-02-13 01:23 Chris Graham Description Updated
2019-06-27 18:58 Chris Graham Tag Attached: Roadmap: v11
2019-10-19 02:02 Chris Graham Assigned To => Chris Graham
2019-10-19 02:02 Chris Graham Status Not Assigned => Resolved
2019-10-19 02:02 Chris Graham Resolution open => fixed