Advanced Dynamic Data Masking by Format Preserving Encryption

This post explains how to apply fine-grained dynamic data masking using the Axiomatics Data Access Filter MD (for Multiple Databases), while minimizing changes to applications that consume the data.

When fine-grained dynamic data masking must also preserve referential integrity and value format.

How could we protect the confidentiality of data, restrict its propagation and still be able to use it? Data masking is the technology most commonly used to conceal confidential data; and products like the Axiomatics Data Access Filter (ADAF MD) provide the tools to apply it in a controlled and very detailed way. But dynamic data masking is far from just a function that obfuscates data. Or rather, the use cases that drive the adoption of dynamic data masking (e.g. compliance and test data management) also ask for minimal changes to the applications that process masked data. Then a dynamic data masking solution, to be most useful, needs to comply with several other requirements.

In “Understanding and Selecting Data Masking Solutions: Creating Secure and Useful Data”, Adrian Lane (Analyst and CTO at Securosis) provides a comprehensive list of requirements on data masking. Two of them, format preservation and referential integrity, are of particular importance to guarantee that applications don’t break when operating on masked data. According to Adrian Lane, format preservation ensures that the mask produces “data with the same structure as the original data”, and referential integrity, that relationships between data values are maintained (e.g. “if a credit card number is a primary key, and scrambled as part of data masking, then all instances of that number linked through key pairs must be scrambled identically”).

In a previous blog post, my colleague Olov Marklund discussed how to combine Policy-based Data Filtering with dynamic data masking, achieving what could more generally be called “Policy-based Data Masking”. ADAF MD leverages an XACML-powered  authorization engine, which means that the decision of when and which dynamic data masking function to apply is directly derived from business rules expressed in teh eXtensible Mark-Up Language (XACML). ADAF MD can also apply a wide range of masking functions, from anything as simple as constant replacement to complex functions able to preserve data format and guarantee its referential integrity.

The Dynamic data masking example

In this blog post, I use one such dynamic data masking function to encrypt the credit card details. The encryption is achieved via a call to an external service capable of providing Format Preserving Encryption (FPE). The FPE implementation used in our example is especially interesting because it can guarantee both format preservation and referential integrity.

The sample data to protect looks like below.

SELECT Top 10 [id]
,[owner]
,[region]
,[status]
,[creditcard]
FROM [dbo].[transactions] where owner in (‘Armando’,’Alice’); 

id

owner

region

status

creditcard

1

Armando

Stockholm

declined

2949-9651-4506-7536

9

Armando

Chicago

approved

0203-5205-5704-2640

12

Alice

Madrid

draft

4638-4745-9379-0289

86

Alice

Paris

approved

9086-9653-9396-0310

95

Armando

Oslo

pending

5687-8587-3888-9468

101

Alice

Oslo

declined

2492-9442-5300-5129

102

Alice

Oslo

declined

3521-9862-1443-3954

103

Alice

Oslo

declined

8366-6022-3749-5333

104

Alice

Oslo

declined

1284-1075-9316-1569

105

Alice

Oslo

declined

8847-9314-7945-5640

The setup is a standard ADAF MD installation (see figure below), with a SQL Proxy component placed between the application and the database. The SQL Proxy intercepts all SQL queries that are sent to the database server. It then queries the ADAF MD authorization service, to check whether the intercepted SQL statements  are aligned with corporate business rules as defined by XACML policies and related external attributes (i.e. properties of the application user, the environment, etc.).

Voltage ADAF MD

The ADAF MD authorization service can make very fine-grained decisions to determine whether the user is authorized to access the data rendered by a SQL statement, down to the level of individual cells. If the user is not authorized, a filter either removes unauthorized records or masks out the portions that are out of bounds. In our example, a deny decision invokes a dynamic data masking function which is executed within the database server itself, which in this case is Microsoft SQL Server 2012. User Defined Functions (UDFs), defined within the database server, use the .NET version of the SecureData Simple API to connect to the SecureData server, a secured web service. This server is responsible for key management and distribution. The Simple API obtains the encryption key from the SecureData server and performs format preserving encryption locally within the database server.

For this blog, we defined a Format Preserving Encryption object for Credit Card called ‘CC’ in the server. The configuration encrypts the Credit Card number with alphanumeric characters while preserving the format. The integration with ADAF MD is trivial. You define the <mask> in the ADAF MD configuration to call the UDF that invokes the encryption service. The configuration for the ‘creditcard’ column looks like this:

<dbObject database=”BANKING” schema=”DBO” table=”TRANSACTIONS” column=”CREDITCARD”>
<mask>banking.dbo.FPEProtect(‘CC’,’****@axiomatics.com’, ‘****’, null, null, creditcard)</mask>
</dbObject>

The UDF FPEProtect has been defined in the database “banking” and schema “dbo”. The first argument of the UDF is the FPE object identifier created in the server. The next two arguments are the identity of the encryptor and the shared key. The last argument is the value/column to be encrypted.

As we saw in the previous blog by Olov, the masking feature can be controlled by XACML policies of the data access filter. Here we define the business policy as follows:

  1. No one can see approved transactions

  2. Managers can see all the transactions

  3. Users can see transactions in their own  regions only

  4. Credit-card details are shown in clear only if the user owns the transaction

With the new way of authoring policies using the Policy Editor in APS 6, it is easy to write the above business rules to single XACML policy. Once the SFS is configured with the policy and attribute connectors to look up the user details from the below table, the ADAF MD authorization service ensures credit card data is encrypted for unauthorized users.

The users table contains details about the following users:

id

role

region

Alice

customer representative

Madrid

Amity

teller

Paris

Andrew

teller

Madrid

Armando

manager

Stockholm

Ava

teller

London

Below is the result set when the query is executed by Armando, a manager:

id

owner

region

status

creditcard

1

Armando

Stockholm

declined

2949-9651-4506-7536

12

Alice

Madrid

draft

4638-AZ8L-5L32-8KAW

95

Armando

Oslo

pending

5687-8587-3888-9468

101

Alice

Oslo

declined

2492-FTUM-5QCK-JA3E

102

Alice

Oslo

declined

3521-KHTC-4AU6-16SG

103

Alice

Oslo

declined

8366-21H1-RSTC-H9YC

104

Alice

Oslo

declined

1284-0MEA-XIDX-SVEO

105

Alice

Oslo

declined

8847-FC5Z-40PF-O8LE

106

Alice

Oslo

declined

7838-EX45-00SU-6MYO

107

Alice

Oslo

declined

9046-H978-HWDX-KJBU

Since Armando is a manager, he can view all the non-approved transactions. The row 1 and 3 are owned by him. Hence the credit card details are in clear. The rest of the credit card details are encrypted with the first four digits preserved and all the dashes preserved while the rest is replaced by alphanumeric strings.

If the same query is executed by Alice, the result set looks like this:

id

owner

region

status

creditcard

12

Alice

Madrid

draft

4638-4745-9379-0289

701

Alice

Madrid

declined

2242-1353-4482-2998

776

Alice

Madrid

draft

9478-9415-1625-7091

1701

Andrew

Madrid

declined

9541-1LV3-D6XX-FAJE

1752

Andrew

Madrid

pending

7118-JALW-2LI2-URLQ

2701

Armando

Madrid

declined

7904-5XCK-6XP8-B4WO

2702

Armando

Madrid

declined

7110-JY1H-ZH5I-MCLQ

2777

Armando

Madrid

draft

7110-JY1H-ZH5I-MCLQ

2778

Armando

Madrid

draft

2665-Q352-W3WS-RYTQ

Since Alice is not a manager, she can view non-approved transactions from her region Madrid. Also, she can view the credit-card details in clear for transactions she owns, the rest are encrypted.

Of course, it is possible to change the encryption format to preserve the last four digits or to use only digits instead of alphanumeric or use ‘*’ instead of digits.

ADAF MD provides the flexibility you need to trigger encryption of data on the fly based on current business rules. A change in business rules will have an immediate effect on how data is filtered and masked without any change whatsoever in applications or databases.

ADAF MD can easily be integrated with third-party services to achieve encryption with minimum efforts. Once the encryption server is setup it takes less than half an hour to integrate ADAF MD with its API.

Other Blogs

3 keys to re-evaluate your authorization management
Business
On May 27, I had the pleasure to join the KuppingerCole KCLive event with several industry peers in a panel discussion about  “Enabling the Future...
How OAuth is related to Attribute Based Access Control
Tech
What is Authorization? Authorization, also referred to as Access Control, is the process that follows authentication (which checks your identity and ensures that you are...
Modern Enterprise Authorization Management System
Business
Gartner has an interesting article titled “Modernize Your Runtime Authorization” that highlights some aspects you need from a modern enterprise authorization systems. Over the years...