A color font file is actually just a regular font file that embeds additional data to display more graphic properties than the contour shapes of a character.
Color fonts are now generally stored as SVG data
inside OpenType font files. This SVG (Scalable Vector Graphics) format can hold vector shapes with color or gradients, and may also include bitmap images - thus leading to bitmap fonts
Ok, the reality is a bit more complex...
The OpenType-SVG font format was initially designed by Mozilla & Adobe and became an industry standard
in early 2016, when other big players including Microsoft & Google agreed on a single format to support color fonts.
All of them (including Apple) have previously developed and implemented their own proprietary color formats to display emojis on their operating systems, while many other companies built other custom color font technologies for the gaming, video or print industries.
Today, there are four major color font formats
that fit into regular font files: SBIX, COLR, CBDT and SVG, each having it own specificities. Read the full story here
or check this simplified recap:
Due to the differences and incompatibilities of these formats (for instance, in early 2017, no operating system supports OpenType-SVG fonts), the design industry will go through another transition period during which several color font formats may be needed to ensure cross-platform compatibility across several operating systems, browsers and apps.
Another solution to mitigate this issue, color fonts can also include some alternate vector shape data as a fallback solution for software that do not yet support any of the embedded color formats.