The _Add Receipt_ form now has a _Restaurant_ toggle. When uploading a
receipt that creates or updates a Firefly III transaction, if the toggle
is activated, a special tag will be added to the transaction. The
assumption is that Firefly will have a rule to automatically adjust the
destination account, category, and/or budget for the transaction if this
tag is present. The tag is configurable and defaults to `Food & Drink`.
dustin/receipts/pipeline/head This commit looks goodDetails
If a new receipt is being attached to an existing Firefly transaction,
we need to update the notes field of the transaction to match what's
specified with the receipt. This was happening correctly for
transactions that already had value in its notes field, but not for
transactions without any notes at all. The reason for this is because
we were only updating the field inside a conditional that checked if the
existing value was not equal to the new value, but it did not account
for the case where there was no existing value at all.
dustin/receipts/pipeline/head This commit looks goodDetails
Unlike the normal `<textarea>` element, `<sl-textarea>` uses a `value`
attribute to store the prefilled value of the field, rather than a child
text node.
Instead of sending the whole image file for every receipt shown on the
list page, we now generate thumbnails for them on the fly. This
dramatically reduces the amount of bytes sent for each image, especially
very large, high-quality photographs. It also improves support for
non-image attachments like PDFs, by rendering image previews in the
grid view instead of a broken image placeholder.
We use GraphicsMagic to do the conversion. Its `MagickWand` API is
pretty straightforward and convenient, and it supports a plethora of
image and image-like formats.
The `transaction` field is usually included in the form submission, even
if it is empty. To avoid querying Firefly for an invalid ID and logging
a useless error about it, we only query if the field is non-empty.
If the value of the `notes` form field is the same as the current value
for the same field of an existing Firefly transaction, we do not need to
update it.
dustin/receipts/pipeline/head This commit looks goodDetails
The Add Receipt form can now create or update transactions in Firefly
III in certain circumstances:
* For existing transactions, if the description, amount, or notes
submitted on the form differ from the corresponding values in Firefly,
the Firefly transaction will be updated with the submitted information
* For gas station transactions, since Chase does not send useful
notifications about these, there is now an option to create an
entirely new transaction in Firefly, using the values provided in the
form
* Similarly for refunds and deposits, which we also do not get helpful
notifications about, the values in the form will be used to create a
new transaction in Firefly
This functionality should help cover most of the edge cases that
`xactmon` cannot handle.
Some of the receipt photos can be quite large. Since there's no
(direct) way to change receipt photos, we can tell the browser to cache
them for a long time to avoid having to download them every time the
list page is shown.
Whenever any of the fields on the page are changed, the form will be
marked as "dirty" until it is submitted. If the form is "dirty" and the
user tries to navigate away from the page, the browser will prompt for
confirmation before leaving the page.
* Reorganizing code into more logical modules:
- `routes` specifically for Rocket handler functions
- `receipts` data model for receipts
- `transactions` for Firefly transactions
* Encapsulate database operations for receipts using the repository
pattern; move SQL queries to external files (`sqlx` can only use
string literals or external files for queries, not variables or
constants)
* Remove obsolete routes, templates for old transaction-focused pages
The Add Receipt form now has a dropdown box that will automatically
populate the data fields from the properties of an existing transaction.
This will be helpful for the cases where the transaction was created
automatically, but still needs a receipt attached to it (e.g. Wal-Mart,
Target, etc. purchases and restaurants).
Rocket makes it easy to dispatch requests for the same path to different
methods based on the `Accept` request header via the `format` argument
to the request attribute.
We'll use this JSON response format to populate a dropdown on the Add
Receipt form, which can be used to automatically fill the fields for
receipt data based on an existing transaction.
A receipt can now be deleted by clicking the little trash can icon on
its card on the receipt list page. To make this look nice, I had to
adjust some of the CSS for that page. Incidentally, I was able to get
the cards to be properly aligned by changing the images to be cropped
instead of scaled, via the `object-fit: cover` CSS property.
This function factors out the logic of extracting an error message from
a `Response` object into a reusable utility function. This will allow
other pages to use the same logic without duplicating it.
If there is no file to be uploaded, then we must not send the value of
the `photo` file input. Doing so causes two files to be uploaded, the
first one being a zero-byte file with no name, and the second one being
the one we captured from the camera. The server only uses the first
uploaded file if there are multiple, so the correct file is never used.
Rethinking the workflow again. Requiring the transaction to be present
in Firefly already will be problematic for two very important cases:
* Gas station purchase never show up in Firefly automatically
* HSA purchase show up hours or days later and have no information
besides the amount
These are arguably the most important cases, since they are the ones
that really need receipts in order to keep the transaction register
correct. Thus, we need a different way to handle these types of
transactions.
Really, what I need is a way to associate transaction data with an image
I really liked the original idea of storing receipts in Paperless, but
that ended up not working out because the OCR failed miserably and thus
made it impossible to search, so finding a receipt meant looking at each
image individually. I think, therefore, the best solution is to store
the images along with manually-entered data.
To implement this new functionality, I am using `sqlx`, a SQL toolkit
for Rust. It's not exactly an ORM, nor does it have a dynamic query
builder like SQLAlchemy, but it does have compile-time checking of
query strings and can produce type-safe query results. Rocket has
support for managing its connection pools as part of the server state,
so that simplifies usage quite a bit.
On the front-end, I factored out the camera image capture into an HTML
custom element, `camera-input`. I did not update the original form to
use it, since I imagine that workflow will actually go away entirely.
Instead of reading our own TOML file for configuration, we can hook into
Rocket's [built-in configuration system][0]. Although it doesn't matter
much right now, it may if we end up using Rocket's SQL [database
integration][1], because it uses the same mechanism. Without making
this change, we would end up with two configuration files in that case.
[0]: https://rocket.rs/guide/v0.5/configuration/#extracting-values
[1]: https://rocket.rs/guide/v0.5/state/#databases
Apparently, you get 640x480 unless you ask for more? Need to constrain
the viewer size, though, otherwise it massively overflows the page and
makes it impossible to see what you're taking a picture of.
It turns out the cropper is _way_ too hard to use on mobile. The
cropper handles are much too small for a touch screen. Maybe I can
figure out how to improve the UX eventually, but for now, Tabitha
requested that we remove that functionality.
Tabitha requested that the application returns to the transaction list
after the form was submitted successfully. Before the navigation
starts, we still want to see the info toast, though. Since the
navigation may take a few seconds, we keep the toast open indefinitely.
It's not immediately clear that submitting the form while the cropper is
active will not include the image _at all_. To make this evident, we
disable the submit button until the crop is complete.
Apparently, Firefox/Android defaults to using the front camera, and
does not provide any native UI for selecting a different one. We can
request the back camera by indiciting that the "environment" camera is
"ideal."
In production deployments, the static assets are stored in
`/usr/local/share/receipts/static`. The working directory is
`/usr/local/share/receipts`, so using a relative path of `static` is
sufficient. We can use the same path in development with a symlink
pointing to the `esbuild` output directory.
And now we come to the meat of the thing: the ability to update
transactions and attach receipts. Most of this is straightforward,
except for changing the amount of split transactions. Hopefully, this
won't come up too often, since I can't really split transactions without
a receipt. Just to be on the safe side, attempting to change the amount
of a split transaction will return an error.
This is all pretty straightforward. The only real problem is that
the search results only contain matching transactions *splits*. Since
transactions themselves do not have an amount, the value shown in the
_Amount_ column on the transaction list may be incorrect if a
transaction contains multiple splits and some of them do not match the
search query.
I've implemented the UI using TypeScript and Shoelace. I originally
started with Pico CSS, but I didn't really like its visuals. Since
capturing photos using the camera requires JavaScript, and that's
basically the entire point of this application, Shoelace's JavaScript
dependency (for WebComponents), is acceptable.
The photo capture uses the Media Capture Web API, which exposes the
camera directly as a video stream. We capture a frame from this stream
and save it in a canvas, which we then pass to Cropper.js to let the
user select only the relevant portion of the picture containing the
receipt itself.
Dustin