Fix off-by-one preventing download of today's data (#264)#318
Conversation
The download date range always stopped at yesterday, so checking today's sleep or stats required waiting until the next day. range(0, (today - date).days) excluded today because both range() and timedelta.days are end-exclusive. Adding + 1 to the three date-delta calculations makes the date range inclusive of today.
|
@tcgoetz I just saw #316 (comment) Can you explain it better? If I'm not missing something, the download already handles overlap downloads from 3 days earlier and the database does the upsert. it shouldn't have duplicated data |
|
I can't give a detailed explanation because its from long ago. I know there was an issue early on and I've never done anything to fix it, so I assume its still there. One thought is that when downloading Garmin Connect data to JSON files, it checks if they exist and doesn't bother downloading if they already exist. We want this behavior for all days except partially days. It saves a lot of downloading and avoids load on the GC API. Anything that was already downloaded for the current day would be partial, but how do you know that you downloaded partial data was downloaded some day in the past? Not downloading partial days at all was an easy solution. So, how do you determine when already downloaded JSON files represent partial data? This will also require a lot of testing across a bunch of data types. |
|
Thanks for the context.
A complete daily file can only have been written after the day it represents ended:
Do you think that would work? The existing download_days_overlap = 3 already handles the common case — if a user syncs The only gap it closes is when a user stops syncing for more than 3 days. In that case, a |
|
That could work. |
The download date range always stopped at yesterday, so checking today's sleep or stats required waiting until the next day.
range(0, (today - date).days) excluded today because both range() and timedelta.days are end-exclusive. Adding + 1 to the three date-delta calculations makes the date range inclusive of today.