Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,219 @@
import { InjectRepository } from '@nestjs/typeorm';

import { Command } from 'nest-commander';
import { ViewType } from 'twenty-shared/types';
import { isDefined } from 'twenty-shared/utils';
import { Repository } from 'typeorm';

import { ActiveOrSuspendedWorkspacesMigrationCommandRunner } from 'src/database/commands/command-runners/active-or-suspended-workspaces-migration.command-runner';
import { RunOnWorkspaceArgs } from 'src/database/commands/command-runners/workspaces-migration.command-runner';
import { ApplicationService } from 'src/engine/core-modules/application/services/application.service';
import { FeatureFlagKey } from 'src/engine/core-modules/feature-flag/enums/feature-flag-key.enum';
import { FeatureFlagService } from 'src/engine/core-modules/feature-flag/services/feature-flag.service';
import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
import { DataSourceService } from 'src/engine/metadata-modules/data-source/data-source.service';
import { PageLayoutType } from 'src/engine/metadata-modules/page-layout/enums/page-layout-type.enum';
import { GlobalWorkspaceOrmManager } from 'src/engine/twenty-orm/global-workspace-datasource/global-workspace-orm.manager';
import { computeTwentyStandardApplicationAllFlatEntityMaps } from 'src/engine/workspace-manager/twenty-standard-application/utils/twenty-standard-application-all-flat-entity-maps.constant';
import { WorkspaceMigrationValidateBuildAndRunService } from 'src/engine/workspace-manager/workspace-migration/services/workspace-migration-validate-build-and-run-service';

@Command({
name: 'upgrade:1-19:backfill-page-layouts',
description:
'Backfill RECORD_PAGE page layouts for legacy workspaces and enable the IS_RECORD_PAGE_LAYOUT_EDITING_ENABLED feature flag',
})
export class BackfillPageLayoutsCommand extends ActiveOrSuspendedWorkspacesMigrationCommandRunner {
constructor(
@InjectRepository(WorkspaceEntity)
protected readonly workspaceRepository: Repository<WorkspaceEntity>,
protected readonly twentyORMGlobalManager: GlobalWorkspaceOrmManager,
protected readonly dataSourceService: DataSourceService,
private readonly applicationService: ApplicationService,
private readonly workspaceMigrationValidateBuildAndRunService: WorkspaceMigrationValidateBuildAndRunService,
private readonly featureFlagService: FeatureFlagService,
) {
super(workspaceRepository, twentyORMGlobalManager, dataSourceService);
}

override async runOnWorkspace({
workspaceId,
options,
}: RunOnWorkspaceArgs): Promise<void> {
const isDryRun = options.dryRun ?? false;

this.logger.log(
`${isDryRun ? '[DRY RUN] ' : ''}Starting backfill of page layouts for workspace ${workspaceId}`,
);

const isAlreadyEnabled = await this.featureFlagService.isFeatureEnabled(
FeatureFlagKey.IS_RECORD_PAGE_LAYOUT_EDITING_ENABLED,
workspaceId,
);

if (isAlreadyEnabled) {
this.logger.log(
`IS_RECORD_PAGE_LAYOUT_EDITING_ENABLED already enabled for workspace ${workspaceId}, skipping`,
);

return;
}
Comment on lines +49 to +59
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The backfill command lacks proper idempotency. It only checks a feature flag set at the end, risking duplicate creation errors on re-run after a partial failure.
Severity: HIGH

Suggested Fix

Before attempting to create entities, add checks to see if they already exist for the current workspace. This can be done by querying for the pageLayout or other core entities using their universalIdentifier. This provides a more robust idempotency safeguard than relying only on a feature flag set at the end of the process.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.

Location:
packages/twenty-server/src/database/commands/upgrade-version-command/1-19/1-19-backfill-page-layouts.command.ts#L48-L59

Potential issue: The `1-19-backfill-page-layouts.command.ts` migration command is not
fully idempotent. It relies solely on the `IS_RECORD_PAGE_LAYOUT_EDITING_ENABLED`
feature flag, which is enabled only after the migration logic completes. If the command
fails after creating database entities but before setting the flag, a subsequent run
will pass the flag check and attempt to re-create the same entities. This will cause a
`UNIQUE` constraint violation on `(workspaceId, universalIdentifier)` for
`PageLayoutEntity` and related entities, leading to migration failure.

Did we get this right? 👍 / 👎 to inform future reviews.


if (isDryRun) {
this.logger.log(
`[DRY RUN] Would create RECORD_PAGE page layouts and enable IS_RECORD_PAGE_LAYOUT_EDITING_ENABLED for workspace ${workspaceId}`,
);

return;
}

const { twentyStandardFlatApplication } =
await this.applicationService.findWorkspaceTwentyStandardAndCustomApplicationOrThrow(
{ workspaceId },
);

const { allFlatEntityMaps: standardAllFlatEntityMaps } =
computeTwentyStandardApplicationAllFlatEntityMaps({
shouldIncludeRecordPageLayouts: true,
now: new Date().toISOString(),
workspaceId,
twentyStandardApplicationId: twentyStandardFlatApplication.id,
});

const recordPageLayoutUniversalIdentifiers = new Set<string>();

const pageLayoutsToCreate = Object.values(
standardAllFlatEntityMaps.flatPageLayoutMaps.byUniversalIdentifier,
)
.filter(isDefined)
.filter((pageLayout) => {
if (pageLayout.type !== PageLayoutType.RECORD_PAGE) {
return false;
}

recordPageLayoutUniversalIdentifiers.add(
pageLayout.universalIdentifier,
);

return true;
});

const tabUniversalIdentifiers = new Set<string>();

const pageLayoutTabsToCreate = Object.values(
standardAllFlatEntityMaps.flatPageLayoutTabMaps.byUniversalIdentifier,
)
.filter(isDefined)
.filter((tab) => {
if (
!recordPageLayoutUniversalIdentifiers.has(
tab.pageLayoutUniversalIdentifier,
)
) {
return false;
}

tabUniversalIdentifiers.add(tab.universalIdentifier);

return true;
});

const pageLayoutWidgetsToCreate = Object.values(
standardAllFlatEntityMaps.flatPageLayoutWidgetMaps.byUniversalIdentifier,
)
.filter(isDefined)
.filter((widget) =>
tabUniversalIdentifiers.has(widget.pageLayoutTabUniversalIdentifier),
);

const viewUniversalIdentifiers = new Set<string>();

const viewsToCreate = Object.values(
standardAllFlatEntityMaps.flatViewMaps.byUniversalIdentifier,
)
.filter(isDefined)
.filter((view) => {
if (view.type !== ViewType.FIELDS_WIDGET) {
return false;
}

viewUniversalIdentifiers.add(view.universalIdentifier);

return true;
});

const viewFieldsToCreate = Object.values(
standardAllFlatEntityMaps.flatViewFieldMaps.byUniversalIdentifier,
)
.filter(isDefined)
.filter((viewField) =>
viewUniversalIdentifiers.has(viewField.viewUniversalIdentifier),
);

const viewFieldGroupsToCreate = Object.values(
standardAllFlatEntityMaps.flatViewFieldGroupMaps.byUniversalIdentifier,
)
.filter(isDefined)
.filter((viewFieldGroup) =>
viewUniversalIdentifiers.has(viewFieldGroup.viewUniversalIdentifier),
);

const validateAndBuildResult =
await this.workspaceMigrationValidateBuildAndRunService.validateBuildAndRunWorkspaceMigration(
{
allFlatEntityOperationByMetadataName: {
pageLayout: {
flatEntityToCreate: pageLayoutsToCreate,
flatEntityToDelete: [],
flatEntityToUpdate: [],
},
pageLayoutTab: {
flatEntityToCreate: pageLayoutTabsToCreate,
flatEntityToDelete: [],
flatEntityToUpdate: [],
},
pageLayoutWidget: {
flatEntityToCreate: pageLayoutWidgetsToCreate,
flatEntityToDelete: [],
flatEntityToUpdate: [],
},
view: {
flatEntityToCreate: viewsToCreate,
flatEntityToDelete: [],
flatEntityToUpdate: [],
},
viewField: {
flatEntityToCreate: viewFieldsToCreate,
flatEntityToDelete: [],
flatEntityToUpdate: [],
},
viewFieldGroup: {
flatEntityToCreate: viewFieldGroupsToCreate,
flatEntityToDelete: [],
flatEntityToUpdate: [],
},
},
workspaceId,
applicationUniversalIdentifier:
twentyStandardFlatApplication.universalIdentifier,
},
);

if (validateAndBuildResult.status === 'fail') {
this.logger.error(
`Failed to create page layouts:\n${JSON.stringify(validateAndBuildResult, null, 2)}`,
);
throw new Error(
`Failed to create page layouts for workspace ${workspaceId}`,
);
}

await this.featureFlagService.enableFeatureFlags(
[FeatureFlagKey.IS_RECORD_PAGE_LAYOUT_EDITING_ENABLED],
workspaceId,
);

this.logger.log(
`Successfully created page layouts and enabled IS_RECORD_PAGE_LAYOUT_EDITING_ENABLED for workspace ${workspaceId}`,
);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,10 @@ import { TypeOrmModule } from '@nestjs/typeorm';

import { AddMissingSystemFieldsToStandardObjectsCommand } from 'src/database/commands/upgrade-version-command/1-19/1-19-add-missing-system-fields-to-standard-objects.command';
import { BackfillMessageChannelMessageAssociationMessageFolderCommand } from 'src/database/commands/upgrade-version-command/1-19/1-19-backfill-message-channel-message-association-message-folder.command';
import { BackfillPageLayoutsCommand } from 'src/database/commands/upgrade-version-command/1-19/1-19-backfill-page-layouts.command';
import { BackfillSystemFieldsIsSystemCommand } from 'src/database/commands/upgrade-version-command/1-19/1-19-backfill-system-fields-is-system.command';
import { ApplicationModule } from 'src/engine/core-modules/application/application.module';
import { FeatureFlagModule } from 'src/engine/core-modules/feature-flag/feature-flag.module';
import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
import { DataSourceModule } from 'src/engine/metadata-modules/data-source/data-source.module';
import { WorkspaceMetadataVersionModule } from 'src/engine/metadata-modules/workspace-metadata-version/workspace-metadata-version.module';
Expand All @@ -23,16 +25,19 @@ import { WorkspaceMigrationModule } from 'src/engine/workspace-manager/workspace
WorkspaceMigrationRunnerModule,
ApplicationModule,
WorkspaceMigrationModule,
FeatureFlagModule,
],
providers: [
BackfillSystemFieldsIsSystemCommand,
AddMissingSystemFieldsToStandardObjectsCommand,
BackfillMessageChannelMessageAssociationMessageFolderCommand,
BackfillPageLayoutsCommand,
],
exports: [
BackfillSystemFieldsIsSystemCommand,
AddMissingSystemFieldsToStandardObjectsCommand,
BackfillMessageChannelMessageAssociationMessageFolderCommand,
BackfillPageLayoutsCommand,
],
})
export class V1_19_UpgradeVersionCommandModule {}
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ import { MigrateWorkflowSendEmailAttachmentsCommand } from 'src/database/command
import { MigrateWorkspacePicturesCommand } from 'src/database/commands/upgrade-version-command/1-18/1-18-migrate-workspace-pictures.command';
import { AddMissingSystemFieldsToStandardObjectsCommand } from 'src/database/commands/upgrade-version-command/1-19/1-19-add-missing-system-fields-to-standard-objects.command';
import { BackfillMessageChannelMessageAssociationMessageFolderCommand } from 'src/database/commands/upgrade-version-command/1-19/1-19-backfill-message-channel-message-association-message-folder.command';
import { BackfillPageLayoutsCommand } from 'src/database/commands/upgrade-version-command/1-19/1-19-backfill-page-layouts.command';
import { BackfillSystemFieldsIsSystemCommand } from 'src/database/commands/upgrade-version-command/1-19/1-19-backfill-system-fields-is-system.command';
import { TwentyConfigService } from 'src/engine/core-modules/twenty-config/twenty-config.service';
import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
Expand Down Expand Up @@ -75,6 +76,7 @@ export class UpgradeCommand extends UpgradeCommandRunner {
protected readonly backfillSystemFieldsIsSystemCommand: BackfillSystemFieldsIsSystemCommand,
protected readonly addMissingSystemFieldsToStandardObjectsCommand: AddMissingSystemFieldsToStandardObjectsCommand,
protected readonly backfillMessageChannelMessageAssociationMessageFolderCommand: BackfillMessageChannelMessageAssociationMessageFolderCommand,
protected readonly backfillPageLayoutsCommand: BackfillPageLayoutsCommand,
) {
super(
workspaceRepository,
Expand Down Expand Up @@ -115,6 +117,7 @@ export class UpgradeCommand extends UpgradeCommandRunner {
this.backfillSystemFieldsIsSystemCommand,
this.addMissingSystemFieldsToStandardObjectsCommand,
this.backfillMessageChannelMessageAssociationMessageFolderCommand,
this.backfillPageLayoutsCommand,
];

this.allCommands = {
Expand Down
Loading