Data and Information Flow Hurdles in Affiliate Marketing

Subtitle: About Data Feeds, Web Services and the use of Interfaces (APIs) in Affiliate Marketing.Slow progress is made when it comes to the utilization of data feeds and web services by affiliate networks and also by advertisers. The biggest progress is currently made when it comes to web services, which is a good thing.The use for data feeds and web services by affiliate publishers might be similar to some degree, but there are distinct cases where they differ significantly. Web services cannot replace data feeds entirely, but they can and should replace some of the uses of data feeds today, where web services do a much better job and where data feeds were a poor (but in the past the only) choice to begin with. Third party providers who aggregate and normalize data feeds will also not be replaced by web services and cannot be replaced by offerings from the individual networks either. Each and everything has its own distinct role and purpose, its core strengths and its own weaknesses. The goal should not be to replace one or the other, but to leverage and support the strengths and reduce or eliminate the weaknesses. Advertisers, networks, 3rd party feeds and widget providers and affiliate publishers all need to play their part and the right use and availability of data feeds and web services as interface between the four parties will ensure that everybody gets the most out of it and be winning at the end.Theory: In theory is only the connection between advertisers and affiliate publishers relevant and networks and 3rd party providers are “only” a middleman that should be eliminated. However, this is not the case in reality. Those middle men are needed and have to play a distinct role in many scenarios where a one-on-one communication between advertiser and publisher is impractical or impossible. The only option in those cases would be that the publisher develops himself the interface that is already provided by the networks and 3rd party providers. This is of course possible, but in many cases a waste of resources on the publishers side and a distraction from the publishers own core competencies.

Reality:If you want to use a computer, you don’t have to buy an operating system or software for it. You could write those things yourself and would keep the software companies out of the loop. Except for a few very specific applications is this scenario completely unrealistic and may sound like a funny joke to some. However, this is exactly what we are talking about, if you are talking about the elimination of the network or 3rd party middleman. I created some diagrams to show the flow of data (information) from a merchant and advertiser to the affiliate publisher. I included more than just product data, because those are not the only information that an affiliate needs to get from an advertiser in order to make this all work. Some of the other information is often overlooked unfortunately, although it is at least as equally important as the product data and should be made accessible via feeds and/or web services too.1. One publisher/affiliate has one relationship to a single advertiser/merchant only.2. One publisher has multiple relationships with different advertisers and merchants that each provides product data, URLs, creative’s directly to the affiliate in the same or different formats and methods.3. A network aggregates all the data from multiple advertisers and provides the data in one format and one method to the individual publisher.4. The publisher has relationships with multiple advertisers via multiple networks. The networks aggregate for him the data from the group of merchants each network has a relationship with. The publisher has now the problem again that the different networks provide the data in different formats and via different methods to him.5. 3rd party service and tool providers aggregate the data across multiple networks and/or multiple advertisers and provides them in one normalized format to the publisher via only delivery method.Here is a link to download the full Diagram

The Roles: What you can see from the diagrams is the fact that networks and 3rd party vendors are playing the role of aggregators and data cleansers who provide data from multiple sources and possibly different format and delivery method to the receiver (the publisher or the 3rd party provider) in one normalized format. Networks aggregate and normalize the data of multiple advertisers and 3rd party providers do the same but a level higher, across multiple networks. The goal and result at the end is a one-on-one relationship with a publisher to make it as simple as possible to use the data to promote the advertisers products and services. That is what the affiliate’s core competence should be; reality is that affiliates are spending way too much time and energy on the aggregation and normalization of data from various different sources, in different formats and delivery methods.

Poor Job Done: The affiliate networks did in the past a very poor job when it came to the aggregation and delivery of normalized data to publishers who needed it. There are no standards, little or no documentation and limited to no access to the data by automated means at all. Even the networks own definitions and specifications were and are often too vague and sometimes even violated by themselves, due to the lack of validation of the data that the network received from the advertiser, often due to the lack or too vague specifications of the feed structure and delivery methods.The use of standards in web services technology, like SOAP or REST help to reduce this kind of issues at least for the web service option that is still in its infancy and evolving.The 3rd party aggregators took over and dealt (and still deal) with all that kind of issues for the publishers and provide data to affiliates in clear format or for direct use via widgets and gadgets.Since networks are in many cases the one-on-one connection point for some affiliates, tools and services that are usually provided by the 3rd party services are and should be provided by the networks directly to the publishers.

Perfect World Scenario: In a perfect world (which should be everybody’s goal that you want to reach eventually, without ever getting there, even if you think that you almost made it) things would be like this. To make it easy for merchants (who provide the original data) and for 3rd party aggregators to get data through the network efficiently standards should be defined and used by all players to avoid vague and incomplete proprietary structures, formats and methods — or simply said — Stop reinventing the wheel (and that poorly to top it all off)! This is another good reason for having an industry association (which we do not have today) — the discussion, specification, agreement and documentation of standards like that. Third party vendors should concentrate on the means of delivery to the publisher and provide the highest flexibility and number of delivery methods as possible. Listen to affiliates suggestions and requests and then implement those. Affiliate network could partner with 3rd party vendors to make the same tools and delivery options available to the publishers directly for the kind of publishers who decide to only work with one particular network directly. This would allow networks to concentrate on their core competencies, like the aggregation and normalization of advertiser data, affiliate tracking and fraud detection and supplemental services like commission payments to the affiliates on behalf of the advertiser and/or affiliate management and recruiting services. Make access to the data available without the need to jump through several hoops. Show what you have and provide documentation and clear instructions, without the need for asking for this information. Agree on standards wherever possible to make everybody’s day easier and to free up time that you could spent on improving your core competencies.

Conclusion: All of this is in no conflict with the ideas of a free market and strong completion. The global market is not localized and proprietary and so you shouldn’t be either. I have a lot of ideas for tools and widgets that would be of great help for publishers and even help to expand your current perspective of who a publisher could be today. The problem is that those ideas (where I am probably not the only one who thinks of the same) all require that the data and information flow is efficient and streamlined that we need to spend less time on reoccurring/repetitive problems and implementing workarounds for shortcomings that shouldn’t exist and more time on implementing those ideas and have publisher do what their core competency is, being innovative in the ways they promote the advertisers products and services and reach into markets where the advertiser cannot or does not want to go or not even know that it existed at all.

To the Future: For resources and information about the mess we are having today, see my affiliate data feeds and web services resources at Cumbrowski.com. They help, but do not solve the core problems that we have.  Cheers!   Carsten Cumbrowski @ Cumbrowski.com  Internet Marketer, Blogger and Entrepreneur

About Carsten Cumbrowski

Internet Marketer, Entrepreneur and Blogger. To learn more about me and what I am doing, visit my website and check out the “about” section.

Twitter: ccumbrowski

9 Responses to Data and Information Flow Hurdles in Affiliate Marketing

  1. "The use of standards in web services technology, like SOAP or REST help to reduce this kind of issues at least for the web service option that is still in its infancy and evolving."

    Just check out our new Web Services (RESTful/SOAP APIs) and join our closed beta program. We'd love to receive valuable feedback.

    http://wiki.zanox.com/En:WebServices

    Product standardization is a huge task that global companies like SAP starting to work on. This is not to be expected within the next two years 🙁

  2. Firuze Gokce says:

    Hi Carsten,

    I want to thank you for stressing on very important topics of affiliate marketing industry.

    In my opinion "Standardization of the presentation of data" is the most critical point. Without standardization I am afraid affiliate marketing could not reach to its performance as expected.

  3. Wayne Porter says:

    Anyone familiar with how the shareware industry uses PAD files?

    I have been toying with this idea for a long time…Another conversation with Angel I think, although it would take someone heavy with Ajax and/or DB savvy.

  4. Very valid thought. The role of the 3rd party providers cannot be ignored in providing ready to use data to the affiliates.

  5. Thomas, Zanox offers Web Services since 2005. What are the additions to the new version (except for the option to choose between different formats and protocols)?

    Firuze: thanks for the comment and I did not forget about you. Sorry for not being able to get back to you yet.

    Wayne: tell me about it. The File_ID.diz became a standard all by itself. The difference is that the need for something like this was immense and obvious to all parties that mattered. This is not the case here. There is still education and information necessary to make some of the parties that do matter and are needed, aware of the problems and the benefits for solving those problems.

    Maynak: Yes, also the networks role. I wanted to make sure that folks don’t see 3rd party providers as competition to networks and bring across the notion that one cannot replace the others. Instead they should partner up to do a better job and do it more efficiently.

  6. Wayne Porter says:

    Carsten,

    I have found a developer who can do something better I think- I have been playing with this idea for a long time. I will talk to Angel to see about using it here (so it is economically viable). It would support Revenews and be useful too boot…

    One thing that would be helpful is all the criteria that smart affiliates look at to evaluate a proposition. e.g. are cookies FIFO or otherwise

    Sliding scale or blends available? Datafeeds? Give me some spec ideas Carsten and I will get with my developer (wporter@gmail.com)

    Angel you have my info too. Given the industry is “shifting” we might as well improve as many things as we can.

    -wayne

  7. > Thomas, Zanox offers Web Services since 2005.
    > What are the additions to the new version (except
    > for the option to choose between different
    > formats and protocols)?

    The new web services now contain new protocols (JSON/XML/SOAP), a new message-based security approach and read/writable resources.

    You will be able to watch new advertiser programs coming up and apply to them easily with a “cron job”. There are a lot of further advancements like geo targeting, behavioral targeting and contextual targeting. We also soon provide a complete new set of reports including a graphical engine to create visuals. You can also create new adspaces on the fly etc. You can pretty much automate everything of the publisher business that you want.

  8. peter bordes says:

    Carsten thank you very much for this great post. we need to start looking at standardizing parts of the performance marketing infrastructure like data feeds.

  9. dee says:

    Carsten – Thanks for the great post. Much info for thought.