ITEM

FEEDBACK

Description

Complaints were received from Lagos State and Olams as described below:


Olams sent a complaint about double debit to us on the 9th of July 2020. Further checks revealed that there was just one batch on NIBSSpay on the 9th and two transactions on the 8th and 9th on the NAPS platform. 

On the 8th and 9th of July, LASG initiated 3 payment files totaling N18,770,120.00. Each of the three NIBSSPay batch IDs (NPP_760534, NPP_752751, NPP_752741) can be found on the portal just once but they have multiple credits on the NAPS platform.


Our preliminary review showed that the duplication occurred on the NIBSSPay/NAPS database. Consequently, an immediate recovery was initiated and escalated to Application Support for further review.

Date

8th & 9th July 2020

Status

The excess debit of N2,790,000,000 was recovered and the Olams account credited on the 10th of July 2020.

We have recovered 85.62% for LASG and currently carry an exposure of N2.7m. We are currently following up on the affected banks.


Further review threw up additional multiple payments for LASG (N7.4m) and Parkway (N1.1m). Banks have been engaged on recovery.


Application Support and the development team have concluded on a fix to address the issue of multiple files and this is currently undergoing approval with the change management board.

Root Cause

The error stems from the architecture of the NPP application which occurs when Requests received are assigned a batch ID from the database and transaction forwarded to the payment mode of choice (NAPS or NEFT). 


Upon transmission to NAPS or NEFT, the file details and status are to be written to the database. Occasionally, failures to connect to the database can occur causing the writing process to fail.


This failure (writing to the database) causes the application to repeatedly attempt the process until successful, thereby issuing another and sometimes multiple Batch IDs, as well as forwarding the same to NAPS or NEFT. This leads to cases of duplicate transactions being processed. 

Actions to Forestall Recurrence 

The NIBSSPay/NAPS service was turned off at 4:06 pm yesterday to forestall further multiple files.

Impact
  • Clients with NAPS payment files in progress will not receive feedback. 
  • Newly initiated NAPS requests will remain on “payment in progress” status until the service is restored.

Interim Resolution 
  • Profiling of clients to use the NEFT payment mode.
  • Manual processing of files for clients with an exceptional request.
  • Cancellation of files for clients whose requests have not gotten to the NAPS platform. This will enable them to re-initiate using the NEFT payment mode.
  • Manual generation of the report from the NAPS platform for clients whose NAPS request got to the NAPS platform before the service was turned off.

Recommendations
  1. Deploy a stopgap failsafe fix that would prevent duplicate transactions from being submitted for a single upload file.  This is awaiting change management approval and will be tested on the staging environment before it is finally deployed to the production environment.
  2. Fix the transaction session management that causes missing transaction on the portal. 
  3. Increase the maximum number of pool connections on the DB to ensure that connection threads are available. this has been implemented.
  4. Update the application to close out DB connection when exiting a process.

 

The stopgap failsafe has been tested on QA and will be deployed and tested on the staging environment. Upon receiving a successful approval by the business users, the deployment will be moved to the production environment.