Improve board experience 🖼️

[lucasbordeau]

This PR improves the general UX and DX of boards, by modifying the query effect to only use paged group by queries.

In this PR we implement two more things in the backend for group by queries :

• Fixed ORDER BY in the PARTITION BY sub-query (this wasn't working because it was applied in the main query, so it sorted randomly picked records, which was a correct sort on an incorrect dataset returned by the sub-query)
• Added offset paging in PARTITION BY

Miscellaneous, various bug fixes and improvements along the way :

• Throttled loading of cards to avoid React freeze
• Handling of drag & drop
• Handling of create / delete / update
• Reworked skeleton (the library slows down a lot with hundreds of skeleton for a spinning effect that is hardly noticed)
• Fixed refetch of aggregate queries (I included the new group by aggregates query we use in the existing refetch mechanism)
• Re-trigger queries on filters and sorts changes
• Unselect all record ids when deleting / restoring / detroying
• Fetch only groups that still have records to lighten the group by query.

What remains to be done

This is still a naïve fetch more implementation that will work for a few fetch more rounds, but if you scroll and load say 200 cards per column on a board, React will re-render all 200 cards of each column each time.

We would probably need to virtualize the board with paged queries as we did for the table, this could be done after this PR but seems less urgent.

What's nice is that this new query pattern is well designed for virtualization also, drawing from our experience with table virtualization, and adapted to a multi-column request pattern, like a 2:2 matrix of records, for our boards.

So the remaining work would be to design a UI solution for virtualizing this matrix of records, which could be quite different from our table virtualization mechanism.

[github-actions]

🚀 Preview Environment Ready!

Your preview environment is available at: http://bore.pub:29479

This environment will automatically shut down when the PR is closed or after 5 hours.

[greptile-apps]

Greptile Overview

Greptile Summary

This PR refactors the record board (Kanban view) data fetching architecture to use a unified group-by query with pagination support, replacing the previous per-column loading approach.

• Backend: Added offsetForRecords parameter to group-by queries, enabling offset-based pagination for records within each group. Extended GraphqlQueryParser with getOrderByRawSQL method for proper ORDER BY handling in partition-by queries.

• Frontend Query Layer: Introduced useRecordIndexGroupsRecordsLazyGroupBy hook with a new generateGroupsRecordsGroupByQuery utility to fetch grouped records. Created useRecordIndexGroupCommonQueryVariables to consolidate shared query variables.

• Board State Management: Added new hooks useTriggerRecordBoardInitialQuery and useTriggerRecordBoardFetchMore to handle initial data loading and infinite scroll pagination. New component state atoms track loading, pagination offset, and per-column fetch-more status.

• Drag-and-Drop: Refactored position calculation to use explicit isDroppedAfterList parameter instead of array index inference, improving accuracy when dropping items at list boundaries. New useUpdateDroppedRecordOnBoard hook manages optimistic updates.

• UI Components: Simplified RecordBoard by moving query logic to RecordBoardQueryEffect. Added RecordBoardFetchMoreInViewTriggerComponent for intersection-observer-based infinite scroll. Updated skeleton loaders with static styling.

• Cleanup: Removed RecordIndexBoardDataLoader, RecordIndexBoardColumnLoaderEffect, and useLoadRecordIndexBoardColumn in favor of the new unified approach.

Confidence Score: 4/5

• This PR is a significant architectural refactor with well-structured changes, but the complexity warrants careful testing of edge cases.
• The PR introduces a cleaner architecture for record board data fetching with proper pagination. The code is well-organized with appropriate separation of concerns. Tests have been updated for the new isDroppedAfterList parameter. However, the scope of changes is large (84 files) and touches critical features like drag-and-drop and data loading.
• Pay attention to useTriggerRecordBoardFetchMore.ts (pagination logic), computeNewPositionOfDraggedRecord.ts (position calculation changes), and group-by-with-records.service.ts (backend pagination implementation).

Important Files Changed

File Analysis

Filename	Score	Overview
packages/twenty-front/src/modules/object-record/record-board/hooks/useTriggerRecordBoardInitialQuery.ts	4/5	New hook that triggers initial group-by query for record board. Handles loading state, updates records in store, and manages pagination.
packages/twenty-front/src/modules/object-record/record-board/hooks/useTriggerRecordBoardFetchMore.ts	4/5	New hook for infinite scroll pagination on record board. Fetches more records per group with offset and handles throttling.
packages/twenty-server/src/engine/api/graphql/graphql-query-runner/group-by/services/group-by-with-records.service.ts	4/5	Backend service extended to support offset-based pagination for group-by queries with records. Added offsetForRecords parameter and proper ORDER BY handling.
packages/twenty-front/src/modules/object-record/record-board/components/RecordBoardQueryEffect.tsx	4/5	New effect component that orchestrates initial query and fetch more logic based on query identifier changes and scroll position.
packages/twenty-front/src/modules/object-record/record-drag/hooks/useUpdateDroppedRecordOnBoard.ts	4/5	New hook for handling drag-and-drop record updates on board. Manages record position updates and moves records between groups.
packages/twenty-front/src/modules/object-record/utils/computeNewPositionOfDraggedRecord.ts	4/5	Refactored to use explicit isDroppedAfterList parameter instead of inferring from array index for more accurate position calculation.
packages/twenty-front/src/modules/object-record/record-index/hooks/useRecordIndexGroupsRecordsLazyGroupBy.ts	4/5	New hook providing lazy Apollo query for fetching grouped records with pagination support.
packages/twenty-front/src/modules/object-record/record-aggregate/utils/generateGroupsRecordsGroupByQuery.ts	5/5	New utility to generate GraphQL group-by query with records, including pagination parameters.
packages/twenty-server/src/engine/api/graphql/graphql-query-runner/graphql-query-parsers/graphql-query.parser.ts	4/5	Added getOrderByRawSQL method to generate raw SQL ORDER BY clause for partition-by queries.

Sequence Diagram

sequenceDiagram
    participant User
    participant RecordBoard
    participant RecordBoardQueryEffect
    participant useTriggerRecordBoardInitialQuery
    participant useTriggerRecordBoardFetchMore
    participant useRecordIndexGroupsRecordsLazyGroupBy
    participant Apollo
    participant GroupByWithRecordsService
    participant Database

    User->>RecordBoard: Navigate to Kanban view
    RecordBoard->>RecordBoardQueryEffect: Mount
    RecordBoardQueryEffect->>useTriggerRecordBoardInitialQuery: queryIdentifierHasChanged
    useTriggerRecordBoardInitialQuery->>useRecordIndexGroupsRecordsLazyGroupBy: executeRecordIndexGroupsRecordsLazyGroupBy()
    useRecordIndexGroupsRecordsLazyGroupBy->>Apollo: Query with groupBy + filter
    Apollo->>GroupByWithRecordsService: resolveWithRecords()
    GroupByWithRecordsService->>Database: SELECT with PARTITION BY + ROW_NUMBER
    Database-->>GroupByWithRecordsService: Grouped records
    GroupByWithRecordsService-->>Apollo: CommonGroupByOutputItem[]
    Apollo-->>useRecordIndexGroupsRecordsLazyGroupBy: Query result
    useRecordIndexGroupsRecordsLazyGroupBy-->>useTriggerRecordBoardInitialQuery: Groups with records
    useTriggerRecordBoardInitialQuery->>RecordBoard: upsertRecordsInStore + setRecordIdsForColumn

    User->>RecordBoard: Scroll down
    RecordBoard->>RecordBoardFetchMoreInViewTriggerComponent: inView detected
    RecordBoardFetchMoreInViewTriggerComponent->>RecordBoardQueryEffect: setShouldFetchMore(true)
    RecordBoardQueryEffect->>useTriggerRecordBoardFetchMore: shouldFetchMore && !isFetching
    useTriggerRecordBoardFetchMore->>useRecordIndexGroupsRecordsLazyGroupBy: executeQuery(offsetForRecords: newOffset)
    useRecordIndexGroupsRecordsLazyGroupBy->>Apollo: Query with offset
    Apollo->>GroupByWithRecordsService: resolveWithRecords(offsetForRecords)
    GroupByWithRecordsService->>Database: SELECT WHERE row_number > offset
    Database-->>GroupByWithRecordsService: Next page of records
    GroupByWithRecordsService-->>Apollo: Paginated results
    Apollo-->>useTriggerRecordBoardFetchMore: New records
    useTriggerRecordBoardFetchMore->>RecordBoard: Append to recordIdsByGroup

Loading

[greptile-apps]

_{84 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

[bosiraphael]

LGTM, left a small nitpick

[twenty-eng-sync]

Hey @lucasbordeau! After you've done the QA of your Pull Request, you can mark it as done here. Thank you!