The problem is that parse_price('', 2) returned NULL instead of throwing
and exception: it seems that accessing var[1] of a text[] variable set
to the empty array, {}, returns NULL, and NULL::integer is, of course,
still NULL.
Apparently, this is the only case, until now, that i had an empty
subtotal, and i did not know what to do: should i keep the function as
is and just handle its NULL return, change it to return 0 in that case,
or raise an exception?
The argument for the first two options, to leave it as is or to
return zero, was that it was convenient for me to allow empty strings as
input values, because that’s what i get from an empty <input>; returning
zero would avoid an extra coalesce everywhere the function was used.
The argument in favor to the last option, an exception, was that the
empty string does not represent an integer, nor a “unknown” (NULL)
integer, therefore the function should do the same when i pass in any
other string that does not represent an integer, just as “a.b”.
At the end i went for option three, because it is the one that breaks
fewer expectatives: casting an empty string to integer, or passing
an empty string as the first value to to_number() throw and exception in
PostgreSQL; my function should do the same. Heck, that what **i**
expected it to do because of the casting inside the function.
To still allow empty strings as parameter to compute_new_expense_amount,
the only case so far, i only had to check for that empty string and
convert it to the string representation of zero, so that parse_price
returns the value i want for that function. This, of course, breaks
the same expectatives as returning NULL for to_price, but i think it is
OK in this case because to_price is more general—used in many more
cases—than compute_new_expense_amount, which is only intended for that
HTML form.
Closes#77.
When adding “free-form products” to quotes they do not have a product
ID, but i has coalescing the NULL to zero because product_id is an
integer and can not coalesce a nullable integer to an empty string.
However, that causes problems when trying to create the invoice for that
quote, because it tries to add products that have an ID of 0 and the
foreign key, obviously, fail.
At first i modified NewInvoiceProductArray.EncodeBinary to check for
"0" as well as the empty string, but i realized this was wrong: the
problem was because i gave these products an ID when they do not have
any. And the solution is to cast product_id to a text, which is what
will get converted anyway because i the only thing i do to it is to
store to a string-backed InputForm field.
Closes#73.
This was requested by a potential user, as they want to be able to do
whatever they want to do to these lists with a spreadsheet.
In fact, they requested to be able to export to CSV, but, as always,
using CSV is a minefield because of Microsoft: since their Excel product
is fucking unable to write and read CSV from different locales, even if
using the same exact Excel product, i can not also create a CSV file
that is guaranteed to work on all locales. If i used the non-standard
sep=; thing to tell Excel that it is a fucking stupid application, then
proper applications would show that line as a row, which is the correct
albeit undesirable behaviour.
The solution is to use a spreadsheet file format that does not have this
issue. As far as I know, by default Excel is able to read XLSX and ODS
files, but i refuse to use the artificially complex, not the actually
used in Excel, and lobbied standard that Microsoft somehow convinced ISO
to publish, as i am using a different format because of the mess they
made, and i do not want to bend over in front of them, so ODS it is.
ODS is neither an elegant or good format by any means, but at least i
can write them using simple strings, because there is no ODS library
in Debian and i am not going to write yet another DEB package for an
overengineered package to write a simple table—all i want is to say
“here are these n columns, and these m columns; have a good day!”.
Part of #51.
Since most of PL/pgSQL functions accept a `uuid` domain, we get an error
if the value is not valid, forcing us to return an HTTP 500, as we
can not detect that the error was due to that.
Instead, i now validate that the slug is indeed a valid UUID before
attempting to send it to the database, returning the correct HTTP error
code and avoiding useless calls to the database.
I based the validation function of Parse() from Google’s uuid package[0]
because this function is an order or magnitude faster in benchmarks:
goos: linux
goarch: amd64
pkg: dev.tandem.ws/tandem/numerus/pkg
cpu: Intel(R) Core(TM) i5-6200U CPU @ 2.30GHz
BenchmarkValidUuid-4 36946050 29.37 ns/op
BenchmarkValidUuid_Re-4 3633169 306.70 ns/op
The regular expression used for the benchmark was:
var re = regexp.MustCompile("^[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-4[a-fA-F0-9]{3}-[8|9|aA|bB][a-fA-F0-9]{3}-[a-fA-F0-9]{12}$")
And the input parameter for both functions was the following valid UUID,
because most of the time the passed UUID will be valid:
"f47ac10b-58cc-0372-8567-0e02b2c3d479"
I did not use the uuid package, even though it is in Debian’s
repository, because i only need to check whether the value is valid,
not convert it to a byte array. As far as i know, that package can not
do that.
[0]: https://github.com/google/uuid
I want this button, as well as the submit button, to be on a row below
the filters’ input, especially for quotes and invoices, that have the
most filters and looks weird with the button wedged in. Thus, i added
a <fieldset> around all the filters.
Closes#69
Sometimes, if the space is tight, the browser may break the radio button
and the single-word label for the toggle, which looks very weird.
I did not set nowrap to all radio button because the no wrap would
prevent long labels to break, and i am not sure if there is any such
radio option. I know that for the toggle there is not any.
This works mostly like invoices: i have to “update” the expense form
to compute its total based on the subtotal and the selected taxes,
although in this case i do no need to compute the subtotal because that
is given by the user.
Nevertheless, i added a new function to compute that total because it
was already hairy enough for the dashboard, that also needs to compute
the tota, not just the base, and i wanted to test that function.
There is no need for a custom input type for that function as it only
needs a couple of simple domains. I have created the output type,
though, because otherwise i would need to have records or “reuse” any
other “amount” output type, which would be confusing.\
Part of #68.
Works exactly the same as for expenses, and this is sometimes convenient
for keeping transfer slips from customers and such.
I actually did not know where to add the download from this attachment,
because if add a column to the index it can easily be confused with the
download icon for the actual invoice.
Part of #66.
We only want two statuses for expense: not yet paid (pending), and paid.
Thus, it is a bit different from quotes and invoices, because expenses
do not pass throw the “workflow” of created→sent→{pending,paid}. That’s
way in this case the status field is already in the new expense form,
instead of hidden, and by pending is not equivalent to created but
unpaid (i.e., the same status color).
With the new select field in the form, the file field no longer can
span two columns or it would be alone on the next row.
Closes#67.
This is mostly to have better contrast with the alternative row color
(#EDE95E), because with Firefox’s default link color (#1B6ACB), on Linux
at least, the contrast ratio is 4,5:1.
Closes#62.
This was requested by Oriol; there are no other technical or legal
requirements for this.
I can not simply append the customer name to the file because it could
have characters that are not valid in file name depending on the
operating system, so i have to “slugify” it.
Closes#65
There was no explicit `order by` in the queries that list the products
of quotes and invoices, so PostgreSQL was free to use any order it
wanted. In this case, since was am grouping first by name, the result
was sorted by product name.
This is not an issue in most cases, albeit a bit rude to the user,
except for when the products *have* to in the same order the user
entered them, because they are monthly fees or something like that, that
must be ordered by month _number_, not by their _name_; the user will
usually input them in the correct order they want them on the invoice or
quote.
Sorting by *_product_id does *not* guarantee that they will always be
in insertion order, because the sequence can “wrap”, but i think i am
going to have bigger problems at that point.
Closes#63
Using Orca or similar accessibility tools, it was not possible to
understand what these “menus” were intended for because they had only
icons without any alternative text, thus nothing to speak aloud with.
By default, when using hx-boost, HTMx adds the `show:top` modifier to
`hx-swap`. This is actually the wanted behaviour in these cases,
because it means that you are following a link to a “different” page,
and it makes sense to start from the top.
However, for quote and invoice forms is a hindrance because you are
usually editing the bottom-most product, as new lines are appended,
therefore you are all the time having to scroll down to the line.
Every. Single. Time.
Here i reverted the show:top to just show:false to keep the scroll as
is. I also added the show:bottom to the new product button, because
that way the window show entirely the new line.
I could not use `show🪟bottom`, however, despite appearing in the
documentation, because that, somehow, ended up doing the same as
`show🪟top`. Not sure if a bug or something i did not understand.
Closes#58.
When i wrote the functions to import contact, i already created a couple
of “temporary” functions to validate whether the input given from the
Excel files was correct according to the various domains used in the
relations, so i can know whether i can import that data.
I realized that i could do exactly the same when validating forms: check
that the value conforms to the domain, in the exact same way, so i can
make sure that the value will be accepted without duplicating the logic,
at the expense of a call to the database.
In an ideal world, i would use pg_input_is_valid, but this function is
only available in PostgreSQL 16 and Debian 12 uses PostgreSQL 15.
These functions are in the public schema because initially i wanted to
use them to also validate email, which is needed in the login form, but
then i recanted and kept the same email validation in Go, because
something felt off about using the database for that particular form,
but i do not know why.
Depending on the number of test ran before this test, the sequence could
overlap with the existing ids and fail because the on conflict do update
would update multiple rows.
This allows to import an Excel file exported from Holded, because it is
our own user case. When we have more customers, we will give out an
Excel template file to fill out.
Why XLSX files instead of CSV, for instance? First, because this is the
output from Holded, but even then we would have more trouble with CSV
than with XLSX because of Microsoft: they royally fucked up
interoperability when decided that CSV files, the files that only other
applications or programmers see, should be “localized”, and use a comma
or a **semicolon** to separate a **comma** separated file depending on
the locale’s decimal separator.
This is ridiculous because it means that CSV files created with an Excel
in USA uses comma while the same Excel but with a French locale expects
the fields to be separated by semicolon. And for no good reason,
either.
Since they fucked up so bad, decided to add a non-standard “meta” field
to specify the separator, writing a `sep=,` in the first line, but this
only works for reading, because saving the same file changes the
separator back to the locale-dependent character and removes the “meta”
field.
And since everyone expects to open spreadsheet with Excel, i can not
use CSV if i do not want a bunch of support tickets telling me that the
template is all in a single line.
I use an extremely old version of a xlsx reading library for golang[0]
because it is already available in Debian repositories, and the only
thing i want from it is to convert the convoluted XML file into a
string array.
Go is only responsible to read the file and dump its contents into a
temporary table, so that it can execute the PL/pgSQL function that will
actually move that data to the correct relations, much like add_contact
does but in batch.
In PostgreSQL version 16 they added a pg_input_is_valid function that
i would use to test whether input values really conform to domains,
but i will have to wait for Debian to pick up the new version.
Meanwhile, i use a couple of temporary functions, in lieu of nested
functions support in PostgreSQL.
Part of #45
[0]: https://github.com/tealeg/xlsx
Eventually, we will allow people to upload their own profile images,
but until then we’ll use a “generic” profile icon instead of the closed
icon, that means nothing to users.
We wanted to have the same icon for that menu than for the Account item,
but the one we used until now did not look inside the round button for
the menu, thus we changed the two.
Related to #55.
These two fields are just for information purposes, as Numerus does not
have any way to wire transfer using these, but people might want to keep
these in the contact’s info as a convenience.
Since not every contact should have an IBAN, e.g., customers, and inside
SEPA (European Union and some more countries) the BIC is not required,
they are in two different relations in order to be optional without
using NULL.
For the IBAN i found an already made PostgreSQL module, but for BIC i
had to write a regular expression based on the information i gathered
from Wikipedia, because the ISO standard is not free.
These two parameters for the add_contact and edit_contact functions are
TEXT because i realized that these functions are intended to be used
from the web application, that only deals with texts, so the
ValueOrNil() function was unnecessarily complex and PostreSQL’s
functions were better suited to “convert” from TEXT to IBAN or BIC.
The same is true for EMAIL and URI domains, so i changed their parameter
types to TEXT too.
Closes#54.
We need to have contacts with just a name: we need to assign
freelancer’s quote as expense linked the government, but of course we
do not have a phone or email for that “contact”, much less a VATIN or
other tax details.
It is also interesting for other expenses-only contacts to not have to
input all tax details, as we may not need to invoice then, thus are
useless for us, but sometimes it might be interesting to have them,
“just in case”.
Of course, i did not want to make nullable any of the tax details
required to generate an invoice, otherwise we could allow illegal
invoices. Therefore, that data had to go in a different relation,
and invoice’s foreign key update to point to that relation, not just
customer, or we would again be able to create invalid invoices.
We replaced the contact’s trade name with just name, because we do not
need _three_ names for a contact, but we _do_ need two: the one we use
to refer to them and the business name for tax purposes.
The new contact_phone, contact_web, and contact_email relations could be
simply a nullable field, but i did not see the point, since there are
not that many instances where i need any of this data.
Now company.taxDetailsForm is no longer “the same as contactForm with
some extra fields”, because i have to add a check whether the user needs
to invoice the contact, to check that the required values are there.
I have an additional problem with the contact form when not using
JavaScript: i must set the required field to all tax details fields to
avoid the “(optional)” suffix, and because they _are_ required when
that checkbox is enabled, but i can not set them optional when the check
is unchecked. My solution for now is to ignore the form validation,
and later i will add some JavaScript that adds the validation again,
so it will work in all cases.
This is to avoid the problem that if i mistype `company_id = 1` instead
of `company_id = $1`, like i fixed in ee0b5d0b, at least it won’t work
in testing now, as all IDs have a three digits number.
We have shown the application to a potential user, and they told us that
it would be very useful to have a total in the table’s footer, so that
they can verify the amount with the bank’s extracts.
It would be very unusual to have an expense from a customer, and we do
not have (yet) a name for supplier or whatever it should be here, so i
used the same name we use for the column in the table.
Had to add an `unsafe` function to be able to translate text with HTML
fragments in it, although the fragments are added back with printf
because the login link is actually not translatable.
It turns out i have been **years** doing this wrong: you are supposed to
pass that value as a text, like 'CURRENT_TIMESTAMP', not like the
keyword so that it returns the current timestamp as a timestamptz.
However, i have been doing it wrong because of a bug in previous
versions of pgTAP[0], that did not take into account keywords such as
CURRENT_TIMESTAMP or CURRENT_DATE and was comparing their actual values,
not the names, therefore i thought that i misread the documentation.
Only now have discovered this because Debian 12 upgraded pgTAP version
to 1.2.0.
[0]: https://github.com/theory/pgtap/issues/244
I tried to have a log line that uses the “common” format from Apache,
because i thought that it would help me reuse regexps i have defined for
fail2ban filters and such.
However, it makes no much sense.
For once, i was repeating the date and time: log.Printf already does
that for me.
And, second, i do not need that data in Numerus’ log because i always
run it behind a proxy that _has_ a “common”-formatted log file, so
there is no need for me to repeat all that data again.
What i need is the IP, to know whether remotedAdd() function works as
expected; the method, to check that the override does its job; the path,
to know what resource the browser requested; the response status code,
so that i do not need to open the browser console for that; the response
size, to keep on eye that i do not return a lot of data; and the
total response time, to realize how long my unoptimized SQL queries
slows the application down.
The rest, Apache should do its job and record it in its log file for
fail2ban and whatever i need the logs for in the future.
I need the actual remote address to add fail2ban rules for it, but i
also to not want everyone to be able to fake X-Forward-For HTTP headers.
Which can contain multiple ip addresses, by the way, so i have to get
only the first one, as the others will be the proxies that the request
has been (re)forwarded to.
The same as for invoices: to allow people to have their own numbering
scheme, and for these that start using the program in the middle of the
current year.
jordi fita mas
oriol carbonell pujolàs