This time of year, it's probably Chinese sky lanterns
Most likely, they are Chinese sky lanterns.
These lanterns are a bit like illuminated kites, autonomous decorative hot air balloons, or a flying version of the Mexican traditional luminaria. They are made with rice paper over a bamboo frame. The frame is a series of nested hoops, which has the side effect of making the lantern collapsible. You may have seen similar lanterns used as lampshades or hanging ornaments.
Sky lanterns include a small dish at the bottom, in which a tea light or small can of Sterno is placed. When lit, the heat from the tiny flame fills the paper lantern with warmer air, and liftoff is achieved. The lanterns are beautiful as they float through the sky, which they do by the thousands in China at New Year's as part of the traditional festivities.
The Lantern Festival is part of the Chinese New Year festival traditions. The festivals begin on the first day of the first month of the Chinese calendar, and end on the fifteenth day of the first month with the Lantern Festival. The Lunar New Year begins on January 23rd this year, and the Lantern Festival will technically happen on February 6th.
However, the lanterns are so entertaining, beautiful, and all-around awesome that many people light theirs and send them off prematurely. Later in the year you also have the possibility of pranksters trying to frighten the populace, or people just having a bit of fun with a pretty toy.
Regardless of the reason they are in the sky to begin with, Chinese sky lanterns can easily be misidentified by people unfamiliar with them. It can also be difficult to judge their size, particularly after dark. (Is it a small thing flying close by, or a large thing flying high up in the sky?)