How to solve any lastmile bugs?

Lastmile Debugging

Google sheet link

If the aws_alerts error isn’t LastmileProcessingStuckRows, click “Show Error Logs” on the slack alert to see what issue you’re dealing with.

LastmileProcessingStuckRows

If there’s no separate LastmileProcessingError:

If a row makes it through preprocessing but not processing, then Reeses most likely didn’t pick it up. The easiest solution is to delete the value in the “Date Preprocessed” field in the google sheet for the stuck row(s). This will send the row to Reeses again and it will most likely solve itself. Nick’s middle step runs at every minute ending in 1 (eg. 1:01, 1:11, 1:21 etc.) and the Processing step usually finishes about 30 seconds after that, at which point the Date Processed field gets populated.

If there is a separate LastmileProcessingError:

These are rarer. The most common issue is that ASCI did not generate an asci number. The error message in aws_alerts will say if this is the case. Sometimes, it can be fixed with a rerun (delete the preprocessed date). If that doesn’t work, see if the ref number looks bad in the google sheet (“Airway Bill / Bill of Lading Number” column) and ask Nick about it.

If it’s a processing error that’s not related to the asci number, just leave it to me. These are very rare — I believe the only times I’ve seen them are for failed API calls to JIRA when it’s down or when we hit the email quota limit.

LastmilePreprocessingError

99% of the time, someone either entered a bad date (eg. 01/01/0023) or it’s a bad PDF.

If it’s a bad date, the error message will complain about “out of bounds datetime”. To solve, manually edit the screwed up date in the google sheet.

Otherwise it will be related to the PDF.

1. OSError: The operator uploaded blank PDFs. Confirm by clicking the two download links on the google sheet and they will say “Couldn’t preview file. This file is empty.” Email the operator and ask for the correct docs. “Your pdfs for ref <ref> vessel <vessel> caused your lastmile job to fail. Please send them to me”. Go to the drive folder and either overwrite the blank ones or upload with a new name and swap the ID of your upload for the ID in the google doc link. Make sure to upload the correct files to the respective folders (invoice and awb folders) and make sure the name of the replacement file is the same as the original if you’re going with the overwrite method. The overwrite method sometimes doesn’t work for Nandy because she occasionally enters duplicate jobs in this scenario to fix it herself which uploads duplicate blank files which prevents you from overwriting the file automatically. Just swap the IDs if it’s Nandy.
2. All other errors that have PyPDF2 in the error message: There’s a latin-1 character or weird PDF formatting going on. The easiest way to fix it is to just use adobe to compress the PDF and then reupload it. Usually it’s only one of the PDFs that’s causing issues. You can either compress and reupload both or you can run the code attached to this message to check which one it is and just compress and reupload the problematic one

from PyPDF2 import PdfFileMerger

from PyPDF2.utils import PdfReadError

file_name = 'test'

file1 = '/Users/justinmitchell/Downloads/[ORDR-2918534] [PONR-GC 029]-INVOICE_28-11-2023_15-13-03 - Nandy Daniel-Welch.pdf'

file2 = '/Users/justinmitchell/Downloads/iata_awb_consignee_016-18547141 - Nandy Daniel-Welch.pdf'

all_attachments = [file1, file1]

# all_attachments = [file2, file2]

pdf_attachment = file_name

merger = PdfFileMerger(strict=False)

for attachment in all_attachments:

    merger.append(attachment, import_bookmarks=False)

merger.write(pdf_attachment)

merger.close()

Replace the paths of file1 and file2 with the paths to the two documents you downloaded. If the first execution succeeds, swap the commenting of the “all_attachments” lines. Whichever one fails is the one that needs to be compressed and reuploaded.