Skip to content

fix: timestamp parsing for RTDB events#273

Merged
ajperel merged 3 commits intomainfrom
@invertase/fix-timestamp-parsing
May 7, 2026
Merged

fix: timestamp parsing for RTDB events#273
ajperel merged 3 commits intomainfrom
@invertase/fix-timestamp-parsing

Conversation

@CorieW
Copy link
Copy Markdown
Member

@CorieW CorieW commented Apr 30, 2026

Resolves #257

Fixes RTDB event timestamp parsing by centralizing ISO 8601 handling and supporting timestamps without microsecond precision.

Adds coverage for the affected timestamp formats to make sure they keep working.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors timestamp parsing in Firebase Database functions to use a centralized utility, improving support for various ISO 8601 formats, including those without microsecond precision. It updates db_fn.py to use timestamp_conversion, refactors get_precision_timestamp in util.py for better robustness, and adds comprehensive test cases. Feedback includes correcting the docstring for get_precision_timestamp to reflect its actual return type and improving the error handling in util.py by providing a more descriptive error message and using the walrus operator for conciseness.

Comment thread src/firebase_functions/private/util.py
Comment thread src/firebase_functions/private/util.py Outdated
@CorieW CorieW requested a review from a team April 30, 2026 12:19
Copy link
Copy Markdown
Contributor

@cabljac cabljac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm, solid

@CorieW CorieW marked this pull request as ready for review April 30, 2026 14:59
Copy link
Copy Markdown

@ajperel ajperel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the logic in this is solid, but as I got help from AI reviewing it suggested we could fix a long-standing but and simply the code at the same time. Let me know what you think?

Comment thread src/firebase_functions/private/util.py Outdated
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not strictly in this PR but as I used AI to review this PR it called out a bug here:

It aggressively strips Z/z and then hardcodes tzinfo=_dt.timezone.utc. If Firebase ever delivers a NANOSECONDS timestamp containing a numeric offset (e.g., 2023-01-01T12:34:56.123456789+05:30), nanosecond_str becomes 123456789+05:30. The code slices [:6] correctly to get 123456 microseconds, but then it forces tzinfo=utc without shifting the hours, meaning the parsed datetime will be wrong by 5 hours and 30 minutes.

Note: Firebase commonly sends Z for nanosecond timestamps, which is why this might not be breaking in production right now, but it's a clear correctness gap compared to the microsecond path which handles offsets properly.

Do you know if we ever don't send 'Z'? Could be good to fix this too?

More generally the same AI agent suggested drastically simplifying the datetime code, which as a side effect would fix this bug by replacing the enum and all the conversion methods with:

def normalize_timestamp_string(time: str) -> str:
    """Truncates sub-second digits to a maximum of 6 (microseconds) to allow standard strptime parsing."""
    if "." not in time:
        return time
    
    prefix, suffix = time.split(".", 1)
    # suffix contains digits followed by timezone, e.g., "123456789Z" or "123456+05:30"
    match = _re.match(r"(\d+)(.*)", suffix)
    if not match:
        raise ValueError(f"Invalid timestamp format: {time}")
    
    digits, tz = match.groups()
    if len(digits) > 6:
        digits = digits[:6]  # Truncate nanoseconds to microseconds
        
    return f"{prefix}.{digits}{tz}"

def timestamp_conversion(time: str) -> _dt.datetime:
    """Converts an ISO 8601 timestamp and returns a timezone-aware datetime object."""
    normalized_time = normalize_timestamp_string(time)
    
    # Choose format based on whether it has a sub-second fraction
    if "." in normalized_time:
        return _dt.datetime.strptime(normalized_time, "%Y-%m-%dT%H:%M:%S.%f%z")
    else:
        return _dt.datetime.strptime(normalized_time, "%Y-%m-%dT%H:%M:%S%z")

I'm relatively new to this code but... I think it's right? Do you agree? If so, let's use this opportunity to both fix the bug and simplify things.

Copy link
Copy Markdown
Member Author

@CorieW CorieW May 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thanks. I agree this is worth fixing here.

I simplified the timestamp parsing so we normalize fractional seconds first, then let strptime(..., %z) handle the timezone. That fixes the nanosecond + numeric offset case without needing separate precision-specific parsing paths.

I also removed the now-unused precision enum/helper functions and kept coverage for seconds, microseconds, nanoseconds, Z, and numeric offsets. Focused tests pass.

input:  2025-10-30T21:15:51Z
parsed: 2025-10-30T21:15:51+00:00
offset: 0:00:00

input:  2024-04-10T12:00:00.123456Z
parsed: 2024-04-10T12:00:00.123456+00:00
offset: 0:00:00

input:  2023-01-01T12:34:56.123456789Z
parsed: 2023-01-01T12:34:56.123456+00:00
offset: 0:00:00

input:  2023-01-01T12:34:56.123456789+05:30
parsed: 2023-01-01T12:34:56.123456+05:30
offset: 5:30:00

input:  2024-04-10T12:00:00.123456-0700
parsed: 2024-04-10T12:00:00.123456-07:00
offset: -1 day, 17:00:00

Copy link
Copy Markdown

@ajperel ajperel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for updating, this looks great! I appreciate the little optimizations over the AI suggestions.

@ajperel ajperel merged commit a4eb8fa into main May 7, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ValueError when parsing RTDB timestamps without microseconds in db_fn.py

3 participants