Import: Membership

Related reading

  • Import Operations Guide — the "run it" (operator runbook)

  • This document — the "how" (API surface, data dictionary, processing flow, open items)

No dedicated requirements page exists for Membership import yet — the import inherits from the general Membership domain requirements.

1. Overview

The Membership import bulk-adds members to a MembershipPeriod from an XLSX file. Each row is processed in its own transaction (REQUIRES_NEW) so one failure does not roll back the whole import. Rows orchestrate several entities: a RaceNumber (found or created), a User/Person (looked up or created via identity matching), an optional Order, and the Membership itself.

The HTTP surface is asynchronous from release 2.4: the upload returns HTTP 202 immediately and the caller polls a companion GET endpoint for the result DTO. Unlike the EP and Results imports (which delegate to a single bulk importer via the whole-file bridge), Membership runs row-by-row through the generic async framework. The aggregated result DTO is built on retrieval from the per-row ImportRowResult entries.

2. API

2.1. Endpoint Reference

Endpoint Purpose

PUT /api/memberships/import

Upload an XLSX and start the async import. Returns HTTP 202 Accepted with an ImportJobDTO — the job is queued on the generic async framework and processed row-by-row.

GET /api/memberships/import/{jobId}

Retrieve the MembershipImportResultDTO once the job reaches COMPLETED or FAILED. Built on-demand from persisted ImportRowResult entries. Returns HTTP 409 while still processing; HTTP 404 for unknown ids or wrong importType.

2.2. Async Delivery Mode

The per-entity endpoint is a thin facade over the generic async framework. MembershipRowProcessor runs in row-by-row mode: the framework iterates the workbook, calls processRow(…​) per row, and persists each outcome to ImportRowResult. The GET endpoint aggregates those rows into MembershipImportResultDTO at retrieval time.

PUT /api/memberships/import
  ?periodId={long}                                   (required)
  &orgId={long}                                      (optional — defaults to caller's tenant)
  &sheetIndex={int}                                  (default: 0)
  Content-Type: multipart/form-data
  Body: file=<xlsx>

HTTP/1.1 202 Accepted
Location: /api/memberships/import/{jobId}
{ "identifier": "<jobId>", "status": "PROCESSING", "skipMappingPhases": true, ... }

# operator polls ...

GET /api/memberships/import/{jobId}

HTTP/1.1 200 OK                     # once COMPLETED or FAILED
{ ... MembershipImportResultDTO ... }
HTTP/1.1 409 Conflict               # while still processing

See Import Operations Runbook for the polling procedure.

2.3. Request Body

Multipart form with a single file part containing the XLSX workbook. One sheet per upload; sheetIndex selects which sheet (0-based, default 0).

Query parameters:

  • periodId (required) — target MembershipPeriod; the period’s organisation scopes the import

  • orgId (optional) — override the caller’s tenant; must match the period’s organisation

  • sheetIndex (default 0) — 0-based sheet index within the workbook

2.4. Response: MembershipImportResultDTO

{
  "totalRows": 120,
  "created": 118,
  "skipped": 1,
  "errors": 1,
  "issues": [
    { "lineNumber": 47, "outcome": "SKIPPED",
      "message": "Membership number not supplied" },
    { "lineNumber": 93, "outcome": "ERROR",
      "message": "Identity number does not match existing person" }
  ]
}

Invariant: totalRows == created + skipped + errors.

The issues list contains one entry per row that did not result in CREATED. Each issue carries the source line number, the outcome, and a human-readable message.

The DTO shape is deliberately minimal — just enough for an operator to tell whether the import succeeded, which rows failed, and why. It will be expanded in a follow-up if the operations team needs richer per-row fields (e.g. membership number, identity number).

2.5. What to Look At

After polling returns the DTO, these are the patterns to confirm the import landed the way you expected:

  • Clean first run against a fresh period: created ≈ totalRows, issues = []

  • Partial failure with recoverable rows: inspect issues — each carries a lineNumber pointing back into the source file so the operator can correct and re-run

  • Systematic failure (e.g. all "identity number does not match"): stop, investigate the file’s identity columns, re-run; don’t blindly retry

See Import Operations Runbook for recovery procedures.

3. Column Definitions

The import supports 30 columns, with 2 required fields.

3.1. Required Fields

Column Type Aliases

FIRST_NAME

STRING

FirstName, Name, FN

LAST_NAME

STRING

LastName, Surname, SN, LN

3.2. Person Fields

Column Type Aliases

TITLE

STRING

Title

ID_NUMBER

STRING

ID, IDNumber, IdentityNumber, Passport, IDPassport

ID_TYPE

STRING

IDType (default: NATIONAL)

ID_COUNTRY

STRING (FK)

Country, IDCountry, Nationality (default: ZA)

GENDER

STRING

Gender, Gen, Sex

DOB

DATE

DOB, DateOfBirth, Birthday

CONTACT_NUMBER

STRING

ContactNumber, Cellphone, CellphoneNumber, CellNumber, Telephone, TelephoneNumber

EMAIL

STRING

Email, EmailAddress

USER_KEY

LONG

UserKey

MAIN_MEMBER_ID

STRING

MainMember, MainMemberID

EXTERNAL_ID

LONG

UID, PID

OTHER_NUMBER

STRING

OtherNumber

3.3. Address Fields

Column Type Aliases

ADDRESS

STRING

Address, Address1

TOWN

STRING

Town, City, TownCity, CityTown

POSTAL_CODE

STRING

PostalCode, Code, ZIP

3.4. Membership Fields

Column Type Aliases

MEMBERSHIP_ID

INTEGER

MID, MembershipID

MEMBERSHIP_NUMBER

STRING

Member#, MemberNo, MemberNumber, Membership#, MembershipNo, MembershipNumber

STATUS

STRING

Status, MembershipStatus (default: ACTIVE)

MEMBERSHIP_TYPE_ID

INTEGER (FK)

MembershipType, MembershipTypeID

MEMBERSHIP_PERIOD_ID

INTEGER (FK)

MembershipPeriod, MembershipPeriodID, Period, PeriodID

MEMBERSHIP_CRITERIA_ID

INTEGER (FK)

MembershipCriteria, MembershipCriteriaID, Criteria, CriteriaID

MEMBERSHIP_VALID_FROM

DATE

ValidFrom

MEMBERSHIP_VALID_TO

DATE

ValidTo

3.5. Order Fields

Column Type Aliases

ORDER_ID

INTEGER

OrderID

ORDER_NUMBER

STRING

OrderNumber, ReceiptNumber

ORDER_STATUS

STRING

OrderStatus (default: P for Pending)

ORDER_PRODUCT_ID

INTEGER

Product, ProductID, OrderProduct, OrderProductID

ORDER_PRODUCT_AMOUNT

DECIMAL

ProductAmount, OrderProductAmount, Amount

3.6. Control Fields

Column Type Aliases

FORCE_CHANGE

BOOLEAN

Fix, Force

4. Foreign Key Resolution

Field Target Entity Resolution Strategy

ID_COUNTRY

Country

Lookup by code (ZA, US) or name

MEMBERSHIP_TYPE_ID

MembershipType

Lookup by id within organisation

MEMBERSHIP_PERIOD_ID

MembershipPeriod

Lookup by id; must match periodId query parameter

MEMBERSHIP_CRITERIA_ID

MembershipCriteria

Lookup by id within the resolved period

5. Processing Flow

5.1. Two-Layer Design

MembershipImportXLS (Spreadsheet Layer)
    |
    +--> createRequestDTO()    [Parse cells to DTO]
    |
    +--> extractRow()          [Process single row]
            |
            v
MembershipImportService (Business Logic Layer)
    |
    +--> addMember()           [Orchestrate creation]
            |
            +--> findOrCreateNumber()
            +--> lookupOrCreateUser()
            +--> findOrCreateOrder()
            +--> findOrCreateMembership()

5.2. Transaction Handling

Each row is processed in its own transaction via REQUIRES_NEW propagation:

@Transactional(propagation = Propagation.REQUIRES_NEW)
public MembershipAddResponseDTO addMember(Organisation organisation,
    MembershipAddRequestDTO request, MembershipPeriod period)

This isolates row failures — one bad row does not roll back already-imported rows.

5.3. Row-Level Flow

MembershipImportXLS:
 1. Receive InputStream + orgId + periodId + sheetIndex
 2. Load workbook into memory (XSSFWorkbook)
 3. Lookup Organisation (tenantService.getOrganisation)
 4. Lookup MembershipPeriod (verify belongs to org)
 5. For each row:
    a. Skip row 0 (header row, detect columns)
    b. Check if blank (first 5 cells empty)
    c. Create MembershipAddRequestDTO
    d. Call membershipImportService.addMember()
    e. Collect response → persist as ImportRowResult

MembershipImportService.addMember():
 1. Validate request (period, person data)
 2. Find or create RaceNumber (by id, else by number)
 3. Find or create User (Person)
    - By person id if provided
    - Else by identity details (matchOrAdd)
    - Else via existing number assignment
 4. Infer gender from title if unknown
 5. Set identity country / type based on country code
 6. Update identification details
 7. Assign personal details, save user
 8. Find or create Order
 9. Find or create Membership
 10. Assign person to number
 11. Return response

5.4. Person Resolution Strategy

Priority order:

  1. By Person ID — if person.id is provided on the row, look up directly

  2. By identity details — call personService.matchOrAdd() with name / ID number, resolving across organisations

  3. By number assignment — if the row’s number already has a person assigned, use that person

matchOrAdd supports multi-organisation lookup: a person known to another org is added to the current organisation rather than duplicated.

Identity check after match:

// Verify identity number matches
if(candidate.getIdentityNumber() != null &&
   candidate.getIdentityNumber().equals(personWrapper.getIdentityNumber()))
    return persons.get(0);
else
    throw new MembershipImportException("Identity number does not match");

5.5. Gender Inference

If gender is unknown or not supplied, infer from title:

if (person.getTitle() != null &&
    (person.getGender() == null || person.getGender() == Gender.UNKNOWN)) {
    String title = person.getTitle().replace(".", "").toUpperCase();
    if (title.equals("MS") || title.equals("MRS") || title.equals("MISS"))
        person.setGender(Gender.FEMALE);
    else
        person.setGender(Gender.MALE);
}

5.6. Identity Country → Identity Type

Set identity type from country code:

if (countryCode.equals("ZA")) {
    person.setIdentityType(IdentityType.NATIONAL);
} else {
    person.setIdentityType(IdentityType.PASSPORT);
}

6. Outcome Classification

Each row produces one of three outcomes persisted on ImportRowResult:

Outcome Meaning

CREATED

New Membership created (and possibly new Person / RaceNumber / Order).

SKIPPED

Row not processed — missing required field or pre-validation failure (e.g. membership number not supplied when period does not auto-generate).

ERROR

Exception occurred processing the row; the row’s REQUIRES_NEW transaction rolled back. Subsequent rows continue.

No UPDATED outcome today: unlike EP, Membership does not upsert by an external key. Re-running the same file adds new memberships rather than updating existing ones. See Outstanding #5.

7. Source Code

File Purpose

MembershipImportXLS.java

Spreadsheet parsing: column definitions, row extraction, per-row dispatch to the service layer.

MembershipRowProcessor.java

Row-by-row bridge — parses ImportJob.configJson (periodId, sheetIndex), invokes MembershipImportService.addMember(…​) per row, converts each response into an ImportRowResult. See Async Import Architecture.

MembershipImportService.java

Business logic orchestration: person/number/order handling, transaction isolation.

MembershipResourceEx.java

REST endpoints PUT /import (async upload) and GET /import/{jobId} (result retrieval). The GET endpoint aggregates ImportRowResult rows into MembershipImportResultDTO.

MembershipAddRequestDTO.java

Request payload for member addition.

MembershipAddResponseDTO.java

Response with outcome per row.

MembershipImportResultDTO.java

Aggregated response DTO returned by the GET endpoint.

Location: admin-service/src/main/java/za/co/idealogic/event/admin/

8. Outstanding Items

Tracked informally — open an ADO ticket before acting on any of these.

# Topic Status

1

Reduce redundant Organisation lookups in the import call stack (mirrors EP Outstanding #6)

Open

2

Validate ORDER_STATUS parsing properly (current code uses only the first character)

Open

3

Replace printStackTrace() swallows in the outer process() catch blocks with a thrown ImportException

Open

4

Generate next membership number when not supplied — optional auto-generation when MembershipPeriod.autoGenerateNumbers = true

Open; today addMember throws MembershipImportException("Membership number not supplied")

5

Add upsert-by-MEMBERSHIP_NUMBER (analogous to EP’s upsert-by-registrationId) so re-running an import updates rather than duplicates

Open

6

Review Order lookup strategy — today uses findByOrganisationAndExternalId(org, sourceOrder.getNumber()); the TODO questions whether the order number should map to Order.number rather than Order.externalId

Open

7

Locate the relevant LineItem and update its amount (rather than the commented-out membership.setProductAmount(…​) path)

Open

8

Extract identity-type and gender parsing into PersonService (duplicated in EventParticipantImportXLS; see EP Outstanding #1, #2)

Open

9

Expand MembershipImportResultDTO with richer per-row fields (membership number, identity number) if operator feedback calls for it

Open — deliberately minimal today

9. Related Documentation