I am using an htmx-infused button to remove the payment, but that
button can not have the CSRF token as value, thus i have to send it in a
header.
The removal of payments warrants a functions, instead of just DELETE
(and CASCADE) as i do for payment methods, because i have to adjust the
status of expenses too. Since i already have functions for everything,
it is not worth using triggers just for that.
This actually should be the “payments and receivables” section, however
this is quite a mouthful; a “receivable” is a payment made **to** you,
therefore “payments” is ok.
In fact, there is still no receivables in there, as they should be in
a separate relation, to constraint them to invoices instead of expenses.
It will be done in a separate commit.
Since this section will be, in a sense, sort of simplified accounting,
i needed to introduce the “payment account” concept. There is no way,
yet, for users to add them, because i have to revamp the “tax details”
section, but this commit started to grow too big already.
The same reasoning for the attachment payment slips as PDF to payment:
something i have to add, but not yet in this commit.
This is to help up “sell” the service: people can look around the demo
to see whether it fits them. Of course, everyone should have the same
username in the demo.
We talked about having the username and password displayed above the
form in the template, but i think it makes more sense to give users as
little work as necessary. Plus, that means i do not have to write them
down while developing.
Whether the database is demo or not is not something that directly
depends on the environment, but rather on which database we are
connected to, thus an environment variable would not make much sense—it
has to be something of the database.
PostgreSQL has no PRAGMA application_id or PRAGMA user_version as with
SQLite to include application-specific values to the database. The
equivalent would be customized options[0], intended for modules
configuration, but that would require me to execute an ALTER DATABASE
in demo.sql with an specific datbase name, or force the use of psql to
run script the script, because then i can use the :DBNAME placeholder.
I guess that the most “standard” way is to just create a function that
returns a know value if the database is demo. Sqitch does not add that
function, therefore it is unlikely to be there by change unless it is
the demo database.
https://www.postgresql.org/docs/15/runtime-config-custom.html
The legal stuff. Required by Spanish law when setting up a site intended
for pecuniary gain, directly or indirectly.
Now we have more pages to the “public web”, and moved the header and
footer from home to the common layout. I also took the opportunity to
change the element from <div> to the appropriate element based on their
use (i.e., <header> and <footer>).
I removed the <div> around the logo because i did not see any use for
it. I may be from a previous design iteration, but it had no style
applied nor any usage at all in JavaScript.
This is for users that belong to more than one company. It is just a
page with links to the home of each company that the user belongs to.
Had to add a second company to the demo data to test it properly, even
though i already have unit tests for multicompany, but, you know….
This works mostly like invoices: i have to “update” the expense form
to compute its total based on the subtotal and the selected taxes,
although in this case i do no need to compute the subtotal because that
is given by the user.
Nevertheless, i added a new function to compute that total because it
was already hairy enough for the dashboard, that also needs to compute
the tota, not just the base, and i wanted to test that function.
There is no need for a custom input type for that function as it only
needs a couple of simple domains. I have created the output type,
though, because otherwise i would need to have records or “reuse” any
other “amount” output type, which would be confusing.\
Part of #68.
Works exactly the same as for expenses, and this is sometimes convenient
for keeping transfer slips from customers and such.
I actually did not know where to add the download from this attachment,
because if add a column to the index it can easily be confused with the
download icon for the actual invoice.
Part of #66.
This allows to import an Excel file exported from Holded, because it is
our own user case. When we have more customers, we will give out an
Excel template file to fill out.
Why XLSX files instead of CSV, for instance? First, because this is the
output from Holded, but even then we would have more trouble with CSV
than with XLSX because of Microsoft: they royally fucked up
interoperability when decided that CSV files, the files that only other
applications or programmers see, should be “localized”, and use a comma
or a **semicolon** to separate a **comma** separated file depending on
the locale’s decimal separator.
This is ridiculous because it means that CSV files created with an Excel
in USA uses comma while the same Excel but with a French locale expects
the fields to be separated by semicolon. And for no good reason,
either.
Since they fucked up so bad, decided to add a non-standard “meta” field
to specify the separator, writing a `sep=,` in the first line, but this
only works for reading, because saving the same file changes the
separator back to the locale-dependent character and removes the “meta”
field.
And since everyone expects to open spreadsheet with Excel, i can not
use CSV if i do not want a bunch of support tickets telling me that the
template is all in a single line.
I use an extremely old version of a xlsx reading library for golang[0]
because it is already available in Debian repositories, and the only
thing i want from it is to convert the convoluted XML file into a
string array.
Go is only responsible to read the file and dump its contents into a
temporary table, so that it can execute the PL/pgSQL function that will
actually move that data to the correct relations, much like add_contact
does but in batch.
In PostgreSQL version 16 they added a pg_input_is_valid function that
i would use to test whether input values really conform to domains,
but i will have to wait for Debian to pick up the new version.
Meanwhile, i use a couple of temporary functions, in lieu of nested
functions support in PostgreSQL.
Part of #45
[0]: https://github.com/tealeg/xlsx
I had to change MethodOverrider to check whether the form is encoded as
multipart/form-data or i would not be able to get the method field from
forms with files.
For now i add the file manually, i.e., outside add_expense and
edit_expense PL/pgSQL functions, because it was faster for me, but i
will probably add an attach_to_expense function, or something like that,
to avoid having the whole ON CONFLICT logic inside Golang—this belongs
to the database.
I actually find more comfortable to select the product from the list
presented up until now, but this is mostly because i have very few
products and the list is not too long, so the idea is that with
JavaScript we will dynamically add an empty product row to the invoice
and then use the name field to search the product by name.
I have the feeling that i am doing something wrong because i ended up
with a lot of HTMx attribute for what i feel is not that much work,
but for now it will work.
I have added the `Is` field to `InputField` in order to include the `id`
attribute to the HTML element, because the HTMLAttributes are attached
to the `input`, not the `div`, and i felt like this one should also be
a custom element based on div, like all the others.
These is not yet any keyboard control to select the search results.
I am not happy with having the search of products in a different URL
than the index, specially since they use the exact same SQL query and
ProductFilter struct, but i did not know how else ask for a different
representation without resorting to the more complicated MIME types.
I use the same pattern as HTMx’s “Click to Edit” example[0], except that
my edit form is triggered by submit and by focus out of the tags input.
I could not, however, use the standard focus out event because it would
also trigger when removing a tag with the mouse, as for a moment the
remove button has the focus and the search input dispatches a bubbling
focusout. I had to resort to a custom event for that, but i am not
happy with it.
The autofocus attribute seems to do nothing in this case, so i need to
manually change the focus to the new input with JavaScript. However,
this means that i can not use the same input ID for all the forms
because getElementById would always return the first in document order,
changing the focus to that same element and automatically submit the
form due to focus out. That’s why in this form i append the invoice’s
slug to the input’s ID.
Finally, this is the first time i am using an HTMx-only solution and i
needed a way to return back just the HTML for the <td>, without <title>,
breadcrumbs, or <dialog>. In principle, the template would be the
“layout”, but then i would need to modify everything to check whether
the template file is empty, or something to that effect, so instead i
created a “standalone” template for these cases.
[0]: https://htmx.org/examples/click-to-edit/
Had to add a new hidden field to the form to know whether, when the
request is HTMx-triggered, to refresh the page, as i do when duplicating
from the index, or redirect the client to the new invoice’s view page,
but only if i was duplicating from that same page, not the index.
Since i now have to target main when redirecting to the view page, so
i had to add a location structure with the required json fields and all
that, when “refreshing” i actually tell HTMx to open the index page
again, which seems faster, now that i am used to boosted links.
I am not sure if, at the end, all pages that now use
mustRenderAppTemplate will be replaced with mustRenderMainTemplate,
but for now i keep them separate to know which routes are already
“boosted”.
I had to change the way /invoices/new and /invoices/batch are handled,
because httprouter was not happy with the new POST /invoices/:slug/edit
route, claiming that /invoices/:slug conflicts with the previously
existing routes.
I also could not make it work with the PATCH method, even though i
correctly added the patchMethod override function, therefore editing
invoices is also weird because i have to take into account the “quick”
invoice status change.
I use the same form for both new and edit invoices, because the only
changes are that we can not edit the invoice date and number, by
Oriol’s design, but must be able to change the status; very similar
forms.
Although it is possible to just print the invoice from the browser, many
people will not even try an assume that they can not create a PDF for
the invoice.
I thought of using Groff or TeX to create the PDF, but it would mean
maintaining two templates in two different systems (HTML and whatever i
would use), and would probably look very different, because i do not
know Groff or TeX that well.
I wish there was a way to tell the browser to print to PDF, and it can
be done, but only with the Chrome Protocol to a server-side running
Chrome instance. This works, but i would need a Chrome running as a
daemon.
I also wrote a Qt application that uses QWebEngine to print the PDF,
much like wkhtmltopdf, but with support for more recent HTML and CSS
standards. Unfortunately, Qt 6.4’s embedded Chromium does not follow
break-page-inside as well as WeasyPrint does.
To use WeasyPrint, at first i wanted to reach the same URL as the user,
passing the cookie to WeasyPrint so that i can access the same invoice
as the user, something that can be done with wkhtmltopdf, but WeasyPrint
does not have such option. I did it with a custom Python script, but
then i need to package and install that script, that is not that much
work, but using the Debian-provided script is even less work, and less
likely to drift when WeasyPrint changes API.
Also, it is unnecessary to do a network round-trip from Go to Python
back to Go, because i can already write the invoice HTML as is to
WeasyPrint’s stdin.
This is not necessary per se, but it makes my life easier because that
way i know which company the user was when she went to its profile and
can “return” back in the menu and future nav items.
At first we thought that a regular text field would do, because we were
afraid that a dropdown would be worse from the point of view of user
experience, but then we realized that we need the country code for VAT
and phone validation, and we can not expect users to input that, of
course.
I had to add the first “i18n table” to the database with the name of all
countries in both Catalan and Spanish and Catalan; English is the
default. For now i think i do not need a view that would select the
name based on the locale of the current request, because currently i do
not plan on adding any other such table —the currency uses the code and
the symbol, thus no need for localization.
However, now i need the language tag from the locale in order to get the
correct translation, and gotext does not give me any way to access the
inner language. Thus the need for our Locale type.
I do not have more time to update the update to the company today, but i
believe this is already a good amount of work for a commit.
The company is going to be used for row level security, as users will
only have access to the data from companies they are granted access, by
virtue of being in the company_user relation.
I did not know how add a row level security policy to the company_user
because i needed the to select on the same relation and this is not
allowed, because it would create an infinite loop.
Had to add the vat, pg_libphonenumber, and uri extensions in order to
validate VAT identification numbers, phone numbers, and URIs,
repectively. These libraries are not in Debian, but i created packages
for them all in https://dev.tandem.ws/tandem.
Had to call xgettext on Go source files because now the title comes from
there, as i assume i will have titles like "Invoice #INVxxxx" that have
to come from the database that the template does not know.
Since users do not have access to the auth scheme, i had to add a view
that selects only the data that they can see of themselves (i.e., no
password or cookie).
I wanted to use the `request.user.id` setting that i set in
check_cookie, but this would be bad because anyone can change that
parameter and, since the view is created by the owner, could see and
*change* the values of everyone just by knowing their id. Thus, now i
use the cookie instead, because it is way harder to figure out, and if
you already have it you can just set to your browser and the user is
fucked anyway; the database can not help here.
I **am** going to use the user id in row level security policies, but
not the value coming for the setting but instaed the one in the
`user_profile`, since it already is “derived” from the cookie, that’s
why i added that column to the view.
The profile includes the language, that i do not use it yet to switch
the locale, so i had to add a relation of the available languages, for
constraint purposes. There is no NULL language, and instead i added the
“Undefined” language, with ‘und’ tag’, to represent “do not know/use
content negotiation”.
The languages in that relation are the same i used to have inside
locale.go, because there is no point on having options for languages i
do not have the translation for, so i now configure the list of
available languages user in content negotiation from that relation.
Finally, i have added all font from RemixIcon because that’s what we
used in the design and i am going to use quite a lot of them.
There is duplication in the views; i will address that in a different
commit.
I did not like the idea that it was the Go server who should set values
such as request.user or set the role, because this is mostly something
that only the database wants for itself, such as when calling logout. I
am also planning to use these setings for row security with the user’s
id, that the Go application has no need for, but with the current
approach i would need to return it from check_cookie so that it can
return it back to the database when acquiring the connection.
I would have used the same function to set the settings and the role,
but security definer functions—obviously in retrospect—can not set the
role, because then could switch to any role of the user that defined the
function, not the roles they are member of. Thus, a new function.
I did not want to do that every time i needed the database connection
within the same request, because it would perform the same operations
each time—it is the same cookie, afterall—, so new connections are
request scoped and passed along in the context.
I had to choose between [1], [2], and [3].
As far as i could find, [1] is not easy to work with templates[4] and at
the moment is not maintained[5].
Both [2] and [3] use the same approach to be used from within templates:
you have to define a FuncMap with template functions that call the
message catalog. Also, both libraries seems to be reasonably
maintained, and have packages in Debian’s repository.
However, [2] repeats the same mistakes that POSIX did with its
catalogs—using identifiers that are not the strings in the source
language—, however this time the catalogs are written in JSON or YAML!
This, somehow, makes things worse….
[3], the one i settled with, is fine and decently maintained. There are
some surprising things, such as to be able to use directly the PO file,
and that it has higher priority over the corresponding MO, or that the
order of parameters is reversed in respect to gettext. However, it uses
a saner format, and is a lot easier to work with than [3].
The problem, of course, is that xgettext does not know how to find
translatable strings inside the template. [3] includes a CLI tool
similar to xgettext, but is not a drop-in replacement[6] and does not
process templates.
The proper way to handle this would be to add a parser to xgettext, but
for now i found out that if i surround the call to the translation
functions from within the template with parentheses, i can trick
xgettext into believing it is parsing Scheme code, and extracts the
strings successfully—at least, for what i have tried. Had to add the
keyword for pgettext, because Schemed does not have it, but at least i
can do that with command line parameters.
For now i left only Spanish and Catalan as the two available languages,
even though the source text is written in English, because that way i
can make sure i do not leave strings untranslated.
[1]: https://golang.org/x/text
[2]: https://github.com/nicksnyder/go-i18n
[3]: https://github.com/leonelquinteros/gotext
[4]: https://github.com/golang/go/issues/39954
[5]: https://github.com/golang/go/issues/12750
[6]: https://github.com/leonelquinteros/gotext/issues/38
At first i thought that i would need to implement sessions, the ones
that keep small files onto the disk, to know which user is talking to
the server, but then i realized that, for now at least, i only need a
very large number, plus the email address, to be used as a lookup, and
that can be stored in the user table, in a separate schema.
Had to change login to avoid raising exceptions when login failed
because i now keep a record of login attemps, and functions are always
run in a single transaction, thus the exception would prevent me to
insert into login_attempt. Even if i use a separate procedure, i could
not keep the records.
I did not want to add a parameter to the logout function because i was
afraid that it could be called from separate users. I do not know
whether it is possible with the current approach, since the settings
variable is also set by the same applications; time will tell.
The whole application will need the same search_path, so it is wasteful
to do that in each handler.
It is possible to pass the search path as a parameter to the database’s
connection string, but then everyone would need to remember to do that,
and update the configuration in case i add another schema.
Similarly, i need to change the user’s role to match her
permissions—which are not in yet—, but this time i need it each time a
handler requests a connection from the pool, because each time the
connection is returned to the pool i reset the role back to the initial,
that hopefully will be authenticator.
jordi fita mas