Hi Mauel,
I can't help you with installing fonts on Debian. But you may try to
load the fonts from their external location, by writing
\setmainfont[ExternalLocation=<PATH>]{<FILENAME>}
in the preamble. (Replace <PATH> by the path to the directory with the
respective font file and <FILENAME> by the font's file name.
HTH,
S.