PapaParse - How to ensure Chunks processed in sequence. Apex response need to fire the next processing sequence
How can I ensure when the first Chunk is inserted in SF then need to fire the second chunk to make sure of sequential processing? I am able to chunk the records in SF and format accordingly but need to insert records in chunks as I process at the client side in sequence.
Papa.parse(file, { worker: true, chunkSize: 1024 * 5, chunk: function (results) { this.papaRecords = results.data; let lstCCToInsert = []; var obj = {}; obj = this.papaRecords; lstCCToInsert.push(obj); RecordInsertJob({ lstCCToInsert: lstCCToInsert, }) .then(result => { thiss.status = result; // when first Chunk inserted in SF then need to fire second chunk ? }) .catch((error) => { }); }, complete: function () { console.log('Complete'); } });
For that, you should have a queue for the data:
dataInsertQueue = [];
And papaparse should put those records in the queue:
this.dataInsertQueue.push(results.data); this.pumpQueue();
Where pumpQueue() checks if there's any pending work and not already busy:
pendingUpload = false; pumpQueue() { if(!pendingUpload && this.dataInsertQueue.length) { this.uploadToServer(); } }And finally, sending to the server:
uploadToServer() { if(this.dataUploadQueue.length) { this.pendingUpload = true; RecordInsertJob({ lstCCToInsert: this.dataUploadQueue.shift(), }) .then(result => { this.status = result; this.pendingUpload = false; this.pumpQueue(); }) } }In other words, you should have a FIFO (First In, First Out) queue to process the records. You'll need to do some additional cleanup, as presumably the status value should state how many records are in total, successful, and error. I leave this as an exercise to you; it should be possible to figure it out once you've gotten this far. Note: Papa Parse is the fastest in-browser CSV (or delimited text) parser for JavaScript. It is reliable and correct according to RFC 4180, and it comes with these features: Easy to use. Parse CSV files directly (local or over the network) Fast mode (is really fast)