When adding “free-form products” to quotes they do not have a product
ID, but i has coalescing the NULL to zero because product_id is an
integer and can not coalesce a nullable integer to an empty string.
However, that causes problems when trying to create the invoice for that
quote, because it tries to add products that have an ID of 0 and the
foreign key, obviously, fail.
At first i modified NewInvoiceProductArray.EncodeBinary to check for
"0" as well as the empty string, but i realized this was wrong: the
problem was because i gave these products an ID when they do not have
any. And the solution is to cast product_id to a text, which is what
will get converted anyway because i the only thing i do to it is to
store to a string-backed InputForm field.
Closes#73.
This was requested by a potential user, as they want to be able to do
whatever they want to do to these lists with a spreadsheet.
In fact, they requested to be able to export to CSV, but, as always,
using CSV is a minefield because of Microsoft: since their Excel product
is fucking unable to write and read CSV from different locales, even if
using the same exact Excel product, i can not also create a CSV file
that is guaranteed to work on all locales. If i used the non-standard
sep=; thing to tell Excel that it is a fucking stupid application, then
proper applications would show that line as a row, which is the correct
albeit undesirable behaviour.
The solution is to use a spreadsheet file format that does not have this
issue. As far as I know, by default Excel is able to read XLSX and ODS
files, but i refuse to use the artificially complex, not the actually
used in Excel, and lobbied standard that Microsoft somehow convinced ISO
to publish, as i am using a different format because of the mess they
made, and i do not want to bend over in front of them, so ODS it is.
ODS is neither an elegant or good format by any means, but at least i
can write them using simple strings, because there is no ODS library
in Debian and i am not going to write yet another DEB package for an
overengineered package to write a simple table—all i want is to say
“here are these n columns, and these m columns; have a good day!”.
Part of #51.
Since most of PL/pgSQL functions accept a `uuid` domain, we get an error
if the value is not valid, forcing us to return an HTTP 500, as we
can not detect that the error was due to that.
Instead, i now validate that the slug is indeed a valid UUID before
attempting to send it to the database, returning the correct HTTP error
code and avoiding useless calls to the database.
I based the validation function of Parse() from Google’s uuid package[0]
because this function is an order or magnitude faster in benchmarks:
goos: linux
goarch: amd64
pkg: dev.tandem.ws/tandem/numerus/pkg
cpu: Intel(R) Core(TM) i5-6200U CPU @ 2.30GHz
BenchmarkValidUuid-4 36946050 29.37 ns/op
BenchmarkValidUuid_Re-4 3633169 306.70 ns/op
The regular expression used for the benchmark was:
var re = regexp.MustCompile("^[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-4[a-fA-F0-9]{3}-[8|9|aA|bB][a-fA-F0-9]{3}-[a-fA-F0-9]{12}$")
And the input parameter for both functions was the following valid UUID,
because most of the time the passed UUID will be valid:
"f47ac10b-58cc-0372-8567-0e02b2c3d479"
I did not use the uuid package, even though it is in Debian’s
repository, because i only need to check whether the value is valid,
not convert it to a byte array. As far as i know, that package can not
do that.
[0]: https://github.com/google/uuid
I want this button, as well as the submit button, to be on a row below
the filters’ input, especially for quotes and invoices, that have the
most filters and looks weird with the button wedged in. Thus, i added
a <fieldset> around all the filters.
Closes#69
Works exactly the same as for expenses, and this is sometimes convenient
for keeping transfer slips from customers and such.
I actually did not know where to add the download from this attachment,
because if add a column to the index it can easily be confused with the
download icon for the actual invoice.
Part of #66.
This was requested by Oriol; there are no other technical or legal
requirements for this.
I can not simply append the customer name to the file because it could
have characters that are not valid in file name depending on the
operating system, so i have to “slugify” it.
Closes#65
There was no explicit `order by` in the queries that list the products
of quotes and invoices, so PostgreSQL was free to use any order it
wanted. In this case, since was am grouping first by name, the result
was sorted by product name.
This is not an issue in most cases, albeit a bit rude to the user,
except for when the products *have* to in the same order the user
entered them, because they are monthly fees or something like that, that
must be ordered by month _number_, not by their _name_; the user will
usually input them in the correct order they want them on the invoice or
quote.
Sorting by *_product_id does *not* guarantee that they will always be
in insertion order, because the sequence can “wrap”, but i think i am
going to have bigger problems at that point.
Closes#63
We need to have contacts with just a name: we need to assign
freelancer’s quote as expense linked the government, but of course we
do not have a phone or email for that “contact”, much less a VATIN or
other tax details.
It is also interesting for other expenses-only contacts to not have to
input all tax details, as we may not need to invoice then, thus are
useless for us, but sometimes it might be interesting to have them,
“just in case”.
Of course, i did not want to make nullable any of the tax details
required to generate an invoice, otherwise we could allow illegal
invoices. Therefore, that data had to go in a different relation,
and invoice’s foreign key update to point to that relation, not just
customer, or we would again be able to create invalid invoices.
We replaced the contact’s trade name with just name, because we do not
need _three_ names for a contact, but we _do_ need two: the one we use
to refer to them and the business name for tax purposes.
The new contact_phone, contact_web, and contact_email relations could be
simply a nullable field, but i did not see the point, since there are
not that many instances where i need any of this data.
Now company.taxDetailsForm is no longer “the same as contactForm with
some extra fields”, because i have to add a check whether the user needs
to invoice the contact, to check that the required values are there.
I have an additional problem with the contact form when not using
JavaScript: i must set the required field to all tax details fields to
avoid the “(optional)” suffix, and because they _are_ required when
that checkbox is enabled, but i can not set them optional when the check
is unchecked. My solution for now is to ignore the form validation,
and later i will add some JavaScript that adds the validation again,
so it will work in all cases.
We have shown the application to a potential user, and they told us that
it would be very useful to have a total in the table’s footer, so that
they can verify the amount with the bank’s extracts.
It is better that way because it works without JavaScript; if HTMx is
not available, it will just use regulars forms.
The problem is that most of the submit buttons where using formaction
to send the request to a different action, and only one button was the
“real” action. Since i could not pass the formaction to
invoice-product-form template, i have changed the “default” action to
the one with “ancillary” functions.
I have to use a different action to remove for each product because i
can not pass the index to the backend without JavaScript: it only
depends on the button click, that already has a name for the action.
Thus, in a way, i have “merged” the action and the index in a single
name.
There is no point in creating a new invoice without products, thus we
were forcing users to always use the “Add product” button for no reason
other than it was easier for me….
I wanted to add the product inside ServeInvoice, when the slug is “new”,
but then it tried to compute the invoice total without price or quantity
and it failed. Thus, i add that product after it has done the
computation query.
For some reason, i assumed that if the invoice product has and ID, that
is it comes from the database, it must also have a product ID, which is
incorrect, because we allow invoice lines with products not added to the
product relation.
I am using zero to mean “no product ID”, so now that validation has to
include the zero as well.
The product search returns a list of products using its slug as the
“external key”, because i do not want people seeing the id in links,
and the search product list is just a different rendering of the product
index table.
However, now i had two almost identical select queries for product,
one using the product_id and the other its slug, the former intended for
the form to select products using checkboxes—the one non-JavaScript
users see—and the latter for the product search.
Using the slug in both forms i can now simplify the code and have a
single query.
With Oriol agreed that adding or editing invoices, products, and
contacts is not just a “user interruption” but the main flow of the
program, and, as such, it is not correct to use dialogs for these.
More importantly, it was harder to concentrate, specially with the more
involved form of invoices, because of all the “noise” behind the dialog.
This is for “free products”, where the user adds an invoice row, does
not select a product from the search, and clicks the update button.
Numerus should select “appropriate” values for those that are left
unspecified.
I also no longer require the product_id to be an integer; if it is
empty, then it is assumed to be a “free product”.
I actually find more comfortable to select the product from the list
presented up until now, but this is mostly because i have very few
products and the list is not too long, so the idea is that with
JavaScript we will dynamically add an empty product row to the invoice
and then use the name field to search the product by name.
I have the feeling that i am doing something wrong because i ended up
with a lot of HTMx attribute for what i feel is not that much work,
but for now it will work.
I have added the `Is` field to `InputField` in order to include the `id`
attribute to the HTML element, because the HTMLAttributes are attached
to the `input`, not the `div`, and i felt like this one should also be
a custom element based on div, like all the others.
These is not yet any keyboard control to select the search results.
I am not happy with having the search of products in a different URL
than the index, specially since they use the exact same SQL query and
ProductFilter struct, but i did not know how else ask for a different
representation without resorting to the more complicated MIME types.
We are going to allow invoices with products that are not (yet) inserted
into the products table.
We always allowed to have products in invoices with a totally different
name, description, price, and whatnot, but until now we had the product
id in these invoice lines for statistics purposes.
However, Oriol raised the concern that this requires for the products
to be inserted before we can create an invoice with them, and we do not
plan to have a “create product while invoicing” feature, thus it would
mean that people would need to cancel the new invoice, create the new
product, and then start the invoice again from scratch.
The compromise is to allow products in the invoice that do not have a
product_id, meaning that at the time the invoice was created they were
not (yet) in the products table. Oriol sees this stop-invoice-create-
product issue more important than the accurate statistics of product
sales, as it will probably be only one or two units off, anyway.
I did not want to allow NULL values to the invoice product’s product_id
field, because NULL means “dunno” instead of “no product”, so i had to
split that field to a separate table that relates an invoice product
with a registered product.
We have reconsidered the toggle thing and instead moved the selection
into a little menu on top of the input, like the input’s label does à
la Material Design.
I just moved the checkboxes into a new details, that works as a menu,
but i had to add the type="search" to the existing input in the tags
field, or the CSS would style the checkboxes as well.
I do not do anything when the checkbox selection changes because that
already triggers a POST to the server that returns the new HTML with
the checkbox changed, and the JavaScript only has to retrieve that new
structure, exactly as it does in the initial rendering.
Since we want to add a little description to the options, i no longer
can use the same SelectOption in ToggleField, even though i could have
reused the Group element, but that felt wrong.
I realized that using a select for just two, short, options is overkill:
the select and its options use a lot more real state than the two
radios, which can have tooltips (not yet, though).
Since i am going to replace this field with a custom element that has
a toggle-like aspect, i already added the is="numerus-toggle" attribute
and use it for stying the non-JavaScript field.
This is because Oriol thinks that there may be cases where you want to
search invoices and such that have any of the selected labels, not all
of them, so we agreed on adding an option to choose.
The idea is that it will be a toggle, but this requires JavaScript and
this commit adds it as a dropdown as a first non-JavaScript step.
I use the same pattern as HTMx’s “Click to Edit” example[0], except that
my edit form is triggered by submit and by focus out of the tags input.
I could not, however, use the standard focus out event because it would
also trigger when removing a tag with the mouse, as for a moment the
remove button has the focus and the search input dispatches a bubbling
focusout. I had to resort to a custom event for that, but i am not
happy with it.
The autofocus attribute seems to do nothing in this case, so i need to
manually change the focus to the new input with JavaScript. However,
this means that i can not use the same input ID for all the forms
because getElementById would always return the first in document order,
changing the focus to that same element and automatically submit the
form due to focus out. That’s why in this form i append the invoice’s
slug to the input’s ID.
Finally, this is the first time i am using an HTMx-only solution and i
needed a way to return back just the HTML for the <td>, without <title>,
breadcrumbs, or <dialog>. In principle, the template would be the
“layout”, but then i would need to modify everything to check whether
the template file is empty, or something to that effect, so instead i
created a “standalone” template for these cases.
[0]: https://htmx.org/examples/click-to-edit/
It all started when i wanted to try to filter invoices by multiple tags
using an “AND”, instead of “OR” as it was doing until now. But
something felt off and seemed to me that i was doing thing much more
complex than needed, all to be able to list the tags as a suggestion
in the input field—which i am not doing yet.
I found this article series[0] exploring different approaches for
tagging, which includes the one i was using, and comparing their
performance. I have not actually tested it, but it seems that i have
chosen the worst option, in both query time and storage.
I attempted to try using an array attribute to each table, which is more
or less the same they did in the articles but without using a separate
relation for tags, and i found out that all the queries were way easier
to write, and needed two joins less, so it was a no-brainer.
[0]: http://www.databasesoup.com/2015/01/tag-all-things.html
This makes reload only the <main> portion of the page, instead of the
whole thing, which to me looks faster; haven’t really measured it.
Like with duplicate, i had to add the location query argument when
inside the view page in order to return back to the same page, not the
index.
Had to add a new hidden field to the form to know whether, when the
request is HTMx-triggered, to refresh the page, as i do when duplicating
from the index, or redirect the client to the new invoice’s view page,
but only if i was duplicating from that same page, not the index.
Since i now have to target main when redirecting to the view page, so
i had to add a location structure with the required json fields and all
that, when “refreshing” i actually tell HTMx to open the index page
again, which seems faster, now that i am used to boosted links.
Changed the invoice number field’s type to search to add the delete icon
on Chromium. Firefox does not add that icon, but i do not care; it is
still better that type="text".
Had to emit the change event to the numerus-tag field, otherwise the
form would not detect the change.
I also can not use keyup as a trigger because the changed modifier can
not be used in the <form>, as nothing ever changes, i do not know how to
trigger the form from children (i.e., data-hx-trigger on the <input>
does nothing), and i can not trigger for just any keyup, or i would
make the request even if they only moved the cursor with the arrow keys,
which is very confusing as Firefox resets the position (this may be due
the fact that i reload the whole <main>, but still).
This is for the unusual case of products without taxes, that PostgreSQL
by default would generate an array with a single null value in it, but
then pgx would not be able to set that null to the string variable.
Initially, this field was meant to be left almost always blank, except
for when we deleted invoiced and had to “replace” its number with a new
invoice; using the automatic numbering in this cas would not “fill in”
the missing number in the sequence.
However, we decide to not allow removing invoicer not edit their
numbers, therefore, if everything goes as planned, there should not be
any gap in the sequence, and that field is rendered useless.
Oriol suggested making it a read-only field, both for new and edit
forms, but i do not think it makes sense to have a field if you can not
edit it at all, specially in the new invoice dialog, where it would
always be blank. In the edit form we already show the number in the
title and breadcrumbs, thus no need for the read-only field as
reference.
I still keep a Number member to the form struct, but is now a string
(kind of “a read-only field”, in a way) and just to be written in the
title or breadcrumbs. I did not like the idea of adding a new SQL
query just for that value.
In this case i have to use the same id for the dialog content in all
pages because, for now, there are a couple of forms that need to replace
it on submit—the new/edit form and the product selection form.
Unfortunately, HTMx does not have support for `formaction` attribute at
this point, so i had to use the workaround described in [0].
[0] https://github.com/bigskysoftware/htmx/issues/623
Instead of using links in the invoice tags, that we will replace with a
“click-to-edit field”, with Oriol agreed to add a form with filters that
includes not only the tags but also dates, customer, status, and the
invoice number.
This means i now need dynamic SQL, and i do not think this belongs to
the database (i.e., no PL/pgSQL function for that). I have looked at
query builder libraries for Golang, and did not find anything that
suited me: either they wanted to manage not only the SQL query but also
all structs, or they managed to confuse Goland’s SQL analyzer.
For now, at least, i am using a very simple approach with arrays, that
still confuses Goland’s analyzer, but just in a very specific part,
which i find tolerable—not that their analyzer is that great to begin
with, but that’s a story for another day.
We agreed with Oriol that this link would only serve to confuse people.
I initially added the link because i thought it was a shame to have to
navigate to the contact section to look or change the info of a customer
that you have an invoice for in front of you. However, it makes little
sense to be able to edit the contact from both sections, and we do not
have a “view page” for contacts to link to, thus the removal.
With Oriol we agreed that contacts should have tags, too, and that the
“tag pool”, as it were, should be shared with the one for invoices (and
all future tags we might add).
I added the contact_tag relation and tag_contact function, just like
with invoices, and then realized that the SQL queries that Go had to
execute were becoming “complex” enough: i had to get not only the slug,
but the contact id to call tag_contact, and all inside a transaction.
Therefore, i opted to create the add_contact and edit_contact functions,
that mirror those for invoice and products, so now each “major” section
has these functions. They also simplified a bit the handling of the
VATIN and phone numbers, because it is now encapsuled inside the
PL/pgSQL function and Go does not know how to assemble the parts.
I am not sure if, at the end, all pages that now use
mustRenderAppTemplate will be replaced with mustRenderMainTemplate,
but for now i keep them separate to know which routes are already
“boosted”.
In that case, strings.Split() return an array with a single empty string
element, that does not pass the domain check for tag_name in the
database.
And an invoice with no tags would get an array of a single NULL in
array_agg, so i had to convert it to an empty string in order for it
to work as expected.
I had to change the way /invoices/new and /invoices/batch are handled,
because httprouter was not happy with the new POST /invoices/:slug/edit
route, claiming that /invoices/:slug conflicts with the previously
existing routes.
I also could not make it work with the PATCH method, even though i
correctly added the patchMethod override function, therefore editing
invoices is also weird because i have to take into account the “quick”
invoice status change.
I use the same form for both new and edit invoices, because the only
changes are that we can not edit the invoice date and number, by
Oriol’s design, but must be able to change the status; very similar
forms.
jordi fita mas