The Day “Big Data” Died…


Long Live Federated Data™…… ( by Tom Noyes with contributions from Russ Schrader)EU Seperation 2

Big news “across the pond” today as the European Union’s highest court struck down the Safe Harbor Agreement. The court offered no grace period for firms to establish their new arrangements before safe harbor ceases to be valid. The court also declined to give extra time to the European Commission, which is currently renegotiating new terms with the U.S. See following stories for more detail:

I tend to view data through a biological, sociological and engineering lens. What systems today know everything about everyone? We all interact and converse based upon our level of trust and objective of conversation. In the engineering sense I only let the space shuttle’s engine know exactly what it needed to do (change in direction) where the controller (Trans DAP) translated that into a gimbal position.  We called these signals. Google calls behaviors it captures signals as well. Searching on the word flu is a signal that you or someone you love may be sick.

From a Biological perspective members of an environment give off many signals, some of which are false, .. but all with a purpose.  What entities in an environment know EVERYTHING about EVERYONE? … The EU is tired of the lax data laws in the environment, most member countries (and their citizens) are MUCH more concerned about privacy than we are here in the US. While at Citibank, I remember that the German regulators made us purge the server log of IP addresses every 30 min.. this for our OWN CUSTOMERS!!?

Whether we agree with the EU or not, consumers, corporations and governments must have methods to control use and transmission of data. The problem is complex, as there are many rights, many “owners” and many requirements (law/regulations) of each environment with jurisdiction over uses and transfers.

Right to have the data

  • Contract with customers, issuers/acquirers
  • Op Regulations of MNOs, Payment Networks, Consortiums
  • Consumer consents
  • US law Privacy/collection restriction (HIPPA, FCRA, COPPA)

Right to share the data

  • GLBA consumer rights/consent/opt out—annual notices that we all get
  • Merchant/Data Owner consents
  • FCRA restrictions—credit, employment, rentals, “character or mode of living”
  • Local/Regional Laws
  • PII issues (Consumer/Object Identifiers)

Right to use the data

  • Use restrictions for sensitive data (Ex GLBA marketing, FCRA not for credit pre-approval, opt out of pre-screening for credit)
  • Onward transfer restricted (GLBA, FCRA)— must have permissible purpose or legitimate interests

What actions should companies take?

  1. Create an inventory of data flows. Know what data flows into and out of every internal business unit, partner, services provider, …etc within EACH regulatory jurisdiction
  2. Map each data flow to the agreement under which it operates
  3. Identify content of data flow, particularly consumer identities and PII
  4. Define and verify the approved USE of that data
  5. Identify HIGH RISK: Uses, Content, Counterparties and Agreements/Gaps 
  6. Develop risk mitigation plan stopping data flow, or restructuring it.
  7. Insert a neutral party to track data flows within a series of  agreements, and/or high risk flows within a company across complex jurisdictions (ex Commerce Signals).

From Big Data to Federated Data™

As an Ex Oracle guy, I know how nice it is to have all of your data in one location. Can you imagine if you gave all of your personal details up to… EVERYTHING!! What you eat everyday.. who you call… what web sites you visit.. Sure they could find you a great spouse….. but… (you fill in the rest of the story).  The same data sharing concerns are happening today with Retailers participating in aggregators asking for all merchant transaction or loyalty or customer data to be onboarded with them as the first step to working together.

Giving up ALL of your data for all of your loyalty card customers can let you do some GREAT things. But…you are losing control and are in essence delegating trust and security of the data  to a third party you can not manage.  My bank can answer the question “does Tom like Italian food?” by looking at the names of restaurants and frequency of dinner visits. Banks could also hand all of their transaction  data over to a third party aggregator to let them manage this… OR they could publish an API “Affinity to X”  and set  a score from 1-10. When someone asks a question, in this sense, the answer comes back as a signal–it basically represents a distributed query.

For internal financial applications (like SAP), I could have every company give me every detail on every customer (which I do need for auditing), or I could ensure my EU based subsidiary has their books in order (trust) and ask their systems for just the items I need to know (ex total sales this week). In a Federated Data world, data holders only provide what is necessary to collaborate within a defined use. If the NSA told us at CommerceSignals “Tom we know that people that watched X movie, went to x restaurant, or travelled to x location are terrorists.. who are they?” I would say to them that I can help you find who can answer your question, but I don’t hold the data… you will have to go to them and ask them yourself. This is the right way to go about privacy and control of data.

The disadvantages of Federated Data? 1) You must ask good questions and can’t take an ad hoc approach to “discovery” of insights. 2) defining the use and purpose of every interaction (with consumer or regulated content) 3) costs of transitioning.

What is Commerce Signals?

Neutral Intermediary that does not store data. We send data to defined destination for a defined use within a defined agreement as directed by the data owner. Commerce Signals’ neutral platform allows businesses to keep data within their own environment and manage data flows within a defined use. Within our Federated Data™ model, exchanges occur within agreements which define the permissions and pricing set by the owner. The Commerce Signals marketplace enables secure collaboration among regulated companies, as well as those with highly sensitive consumer data. Our patent pending signals model protects consumer privacy and data ownership while enabling the creation of great consumer experiences within a highly complex global regulatory environment.

Approach to privacy

Leave a Reply

Your email address will not be published.