When i wrote the functions to import contact, i already created a couple
of “temporary” functions to validate whether the input given from the
Excel files was correct according to the various domains used in the
relations, so i can know whether i can import that data.
I realized that i could do exactly the same when validating forms: check
that the value conforms to the domain, in the exact same way, so i can
make sure that the value will be accepted without duplicating the logic,
at the expense of a call to the database.
In an ideal world, i would use pg_input_is_valid, but this function is
only available in PostgreSQL 16 and Debian 12 uses PostgreSQL 15.
These functions are in the public schema because initially i wanted to
use them to also validate email, which is needed in the login form, but
then i recanted and kept the same email validation in Go, because
something felt off about using the database for that particular form,
but i do not know why.
This allows to import an Excel file exported from Holded, because it is
our own user case. When we have more customers, we will give out an
Excel template file to fill out.
Why XLSX files instead of CSV, for instance? First, because this is the
output from Holded, but even then we would have more trouble with CSV
than with XLSX because of Microsoft: they royally fucked up
interoperability when decided that CSV files, the files that only other
applications or programmers see, should be “localized”, and use a comma
or a **semicolon** to separate a **comma** separated file depending on
the locale’s decimal separator.
This is ridiculous because it means that CSV files created with an Excel
in USA uses comma while the same Excel but with a French locale expects
the fields to be separated by semicolon. And for no good reason,
either.
Since they fucked up so bad, decided to add a non-standard “meta” field
to specify the separator, writing a `sep=,` in the first line, but this
only works for reading, because saving the same file changes the
separator back to the locale-dependent character and removes the “meta”
field.
And since everyone expects to open spreadsheet with Excel, i can not
use CSV if i do not want a bunch of support tickets telling me that the
template is all in a single line.
I use an extremely old version of a xlsx reading library for golang[0]
because it is already available in Debian repositories, and the only
thing i want from it is to convert the convoluted XML file into a
string array.
Go is only responsible to read the file and dump its contents into a
temporary table, so that it can execute the PL/pgSQL function that will
actually move that data to the correct relations, much like add_contact
does but in batch.
In PostgreSQL version 16 they added a pg_input_is_valid function that
i would use to test whether input values really conform to domains,
but i will have to wait for Debian to pick up the new version.
Meanwhile, i use a couple of temporary functions, in lieu of nested
functions support in PostgreSQL.
Part of #45
[0]: https://github.com/tealeg/xlsx
These two fields are just for information purposes, as Numerus does not
have any way to wire transfer using these, but people might want to keep
these in the contact’s info as a convenience.
Since not every contact should have an IBAN, e.g., customers, and inside
SEPA (European Union and some more countries) the BIC is not required,
they are in two different relations in order to be optional without
using NULL.
For the IBAN i found an already made PostgreSQL module, but for BIC i
had to write a regular expression based on the information i gathered
from Wikipedia, because the ISO standard is not free.
These two parameters for the add_contact and edit_contact functions are
TEXT because i realized that these functions are intended to be used
from the web application, that only deals with texts, so the
ValueOrNil() function was unnecessarily complex and PostreSQL’s
functions were better suited to “convert” from TEXT to IBAN or BIC.
The same is true for EMAIL and URI domains, so i changed their parameter
types to TEXT too.
Closes#54.
We need to have contacts with just a name: we need to assign
freelancer’s quote as expense linked the government, but of course we
do not have a phone or email for that “contact”, much less a VATIN or
other tax details.
It is also interesting for other expenses-only contacts to not have to
input all tax details, as we may not need to invoice then, thus are
useless for us, but sometimes it might be interesting to have them,
“just in case”.
Of course, i did not want to make nullable any of the tax details
required to generate an invoice, otherwise we could allow illegal
invoices. Therefore, that data had to go in a different relation,
and invoice’s foreign key update to point to that relation, not just
customer, or we would again be able to create invalid invoices.
We replaced the contact’s trade name with just name, because we do not
need _three_ names for a contact, but we _do_ need two: the one we use
to refer to them and the business name for tax purposes.
The new contact_phone, contact_web, and contact_email relations could be
simply a nullable field, but i did not see the point, since there are
not that many instances where i need any of this data.
Now company.taxDetailsForm is no longer “the same as contactForm with
some extra fields”, because i have to add a check whether the user needs
to invoice the contact, to check that the required values are there.
I have an additional problem with the contact form when not using
JavaScript: i must set the required field to all tax details fields to
avoid the “(optional)” suffix, and because they _are_ required when
that checkbox is enabled, but i can not set them optional when the check
is unchecked. My solution for now is to ignore the form validation,
and later i will add some JavaScript that adds the validation again,
so it will work in all cases.
I have to use a value to be used as “none” for payment method and
contact. In PL/pgSQL add_quote and edit_quote functions, that value is
NULL, while in forms it is the empty string. I can not simply pass the
empty string for either of these fields because PL/pgSQL expects
(nullable) integers, and "" is not a valid integer and is not NULL
either. A conversion is necessary.
Apparently, Go’s nil is not a valid representation for SQL’s NULL with
pgx, and had to use sql.NullString instead.
I also needed to coalesce contact’s VATIN and phone, because null values
can not be scanned to *string. I did not do that before because
`coalesce(vatin, '')` throws an error that '' is not a valid VATIN and
just left as is, wrongly expecting that pgx would do the job of leaving
the string blank for me. It does not.
Lastly, i can not blindly write Quotee’s tax details in the quote’s view
page, or we would see the (), characters for the empty address info.
Otherwise, pgx (rightfully) tries to convert a "" into a integer, as
this is the field’s type, cannot, and panics with an error.
Added a IntegerOrNull method to FormField because this is exactly the
same that happens with the invoiceProductId, and made no sense to have
to do the logic twice, or in a function inside form.
I had to change MethodOverrider to check whether the form is encoded as
multipart/form-data or i would not be able to get the method field from
forms with files.
For now i add the file manually, i.e., outside add_expense and
edit_expense PL/pgSQL functions, because it was faster for me, but i
will probably add an attach_to_expense function, or something like that,
to avoid having the whole ON CONFLICT logic inside Golang—this belongs
to the database.
I actually find more comfortable to select the product from the list
presented up until now, but this is mostly because i have very few
products and the list is not too long, so the idea is that with
JavaScript we will dynamically add an empty product row to the invoice
and then use the name field to search the product by name.
I have the feeling that i am doing something wrong because i ended up
with a lot of HTMx attribute for what i feel is not that much work,
but for now it will work.
I have added the `Is` field to `InputField` in order to include the `id`
attribute to the HTML element, because the HTMLAttributes are attached
to the `input`, not the `div`, and i felt like this one should also be
a custom element based on div, like all the others.
These is not yet any keyboard control to select the search results.
I am not happy with having the search of products in a different URL
than the index, specially since they use the exact same SQL query and
ProductFilter struct, but i did not know how else ask for a different
representation without resorting to the more complicated MIME types.
We have reconsidered the toggle thing and instead moved the selection
into a little menu on top of the input, like the input’s label does à
la Material Design.
I just moved the checkboxes into a new details, that works as a menu,
but i had to add the type="search" to the existing input in the tags
field, or the CSS would style the checkboxes as well.
I do not do anything when the checkbox selection changes because that
already triggers a POST to the server that returns the new HTML with
the checkbox changed, and the JavaScript only has to retrieve that new
structure, exactly as it does in the initial rendering.
Since we want to add a little description to the options, i no longer
can use the same SelectOption in ToggleField, even though i could have
reused the Group element, but that felt wrong.
I realized that using a select for just two, short, options is overkill:
the select and its options use a lot more real state than the two
radios, which can have tooltips (not yet, though).
Since i am going to replace this field with a custom element that has
a toggle-like aspect, i already added the is="numerus-toggle" attribute
and use it for stying the non-JavaScript field.
Instead of using links in the invoice tags, that we will replace with a
“click-to-edit field”, with Oriol agreed to add a form with filters that
includes not only the tags but also dates, customer, status, and the
invoice number.
This means i now need dynamic SQL, and i do not think this belongs to
the database (i.e., no PL/pgSQL function for that). I have looked at
query builder libraries for Golang, and did not find anything that
suited me: either they wanted to manage not only the SQL query but also
all structs, or they managed to confuse Goland’s SQL analyzer.
For now, at least, i am using a very simple approach with arrays, that
still confuses Goland’s analyzer, but just in a very specific part,
which i find tolerable—not that their analyzer is that great to begin
with, but that’s a story for another day.
With Oriol we agreed that contacts should have tags, too, and that the
“tag pool”, as it were, should be shared with the one for invoices (and
all future tags we might add).
I added the contact_tag relation and tag_contact function, just like
with invoices, and then realized that the SQL queries that Go had to
execute were becoming “complex” enough: i had to get not only the slug,
but the contact id to call tag_contact, and all inside a transaction.
Therefore, i opted to create the add_contact and edit_contact functions,
that mirror those for invoice and products, so now each “major” section
has these functions. They also simplified a bit the handling of the
VATIN and phone numbers, because it is now encapsuled inside the
PL/pgSQL function and Go does not know how to assemble the parts.
We only allow a single tax of each class in products and invoices,
because it does not make sense to add two different VAT to the same
product, for instance.
We will only allow to select a tax from each of the tax classes, but
the user needs to know what class each tax belongs to, and grouping
the taxes by class in the select helps with that.
I have moved everything into a different file, even though it is related
to “db”, because it was starting to get a bit ugly.
Apparently i was doing too much work and had to read the code to
understand what i was supposed to do, because pgtypes’ documentation,
as all other projects from the same author, is almost non-existent.
It seems that we do not agree en whether the IRPF tax should be
something of the product or the contact, so we decided to make the
product have multiple taxes, just in case, and if only one is needed,
then users can just select one; no need to limit to one.
This is not yet necessary, but the empty label is because i do not want
to select a default tax for products—at least, not without a setting for
it.
Since i need to add the required attribute now to select, because
otherwise the browser would allow sending that empty value, i did not
want to do it unconditionally, just in case.
I do not want to use floats in the Go lang application, because it is
not supposed to do anything with these values other than to print and
retrieve them from the user; all computations will be performed by
PostgreSQL in cents.
That means i have to “convert” from the price format that users expect
to see (e.g., 1.234,56) to cents (e.g., 123456) and back when passing
data between Go and PostgreSQL, and that conversion depends on the
currency’s decimal places.
At first i did everything in Go, but saw that i would need to do it in
a loop when retrieving the list of products, and immediately knew it was
a mistake—i needed a PL/pgSQL function for that.
I still need to convert from string to float, however, when printing the
value to the user. Because the string representation is in C, but i
need to format it according to the locale with golang/x/text. That
package has the information of how to correctly format numbers, but it
is in an internal package that i can not use, and numbers.Digit only
accepts numeric types, not a string.
I implemented the Valuer and Scanner interfaces to InputField and
SelectField for better passing values between the database and Go. I
had a conflict with the Value name and renamed the struct member to Val.
I also had to change the attributes array to be of type
template.HTMLAttr or html/template would replace `form="newtax"`
attribute to `zgotmplz="newtax"` because it deems it “unsafe”. I do
not like having to use template.HTMLAttr when assigning values, but
i do not know what else i can do now.
I was worried that i was repeating the AddInputErrors function for each
form, because they were basically the same. I could create a Form type
and make all forms embed it, but i realized that with a separate
validator i would have cleaner validation functions and would not need
the Valid field in the form that i am using only for that method.
Similar to the profile form, the login form now parses and validates
itself, with the InputField structs that the templates expect.
I realized that i was doing more work than necessary when parsing fields
fro the profile form because i was repeating the operation and the field
name, so now it is a function of InputField.
This time i needed extra attributes for the login form. I am not sure
that the Go source code needs to know about HTML attributes, but it was
the easiest way to pass them to the template.
Let’s start first with a non-fancy validation method with just if
conditionals instead of bringing yet another complicated library. I
hope i do not regret it.
I wanted to move all the input field to a template because all that
gobbledygook with the .input div and repeating the label in the
placeholder was starting to annoy me. Now with error messages was even
more concerning.
I did not know whether the label should be a part of the input fields
or something that the template should do. At the end i decided that
it makes more sense to be part of the input field because in the error
messages i use that same label, thus the template does not have a say
in that, and, besides, it was just easier to write the template.
The same with the error messages: i’ve seen frameworks that have a map
with the field’s id/name to the error slice, but then it would be
a bit harder to write the template.
I added AddError functions instead of just using append inside the
validator function, and have a local variable for whether it all went
OK, because i was worried that i would leave out the `ok = false`
in some conditions.
I had started writing “constructors” functions for InputField and
SelectField, but then had to add other methods to change the required
field and who knows what else, and in the end it was easier to just
construct the field inline.
jordi fita mas