Input Data Errors

Dear Readers,

I owe you an apology. I made a mistake. I am writing this letter in the first person, because I made the mistake. Let me explain what happened.

 

The wrong stuff went into the funnel in the upper left-hand corner…

 

I wrote software to calculate the gold basis and co-basis (and of course silver too). The app does not just calculate the near contract. It calculates the basis for many contracts out in the distance, so I can see the whole picture.

I developed a model for the fundamental gold price, based on the basis. My software calculates this, too (spoiler alert: the reported fundamental prices were high).

I have long since debugged it. It works reliably. So reliably, that every day I pored over the results, but I no longer checked the inputs and intermediate steps of the calculation. Now, in retrospect, I realize that I should have.

The root cause is simple. For as far back as I have ever seen, the symbol for a future has been a two-letter code for the commodity + a one letter for the month and one digit for the year. For example, gold is GC. December is Z. And 2017 is 7. So the December gold contract is “GC Z7”. Silver is SI, so December silver is “SI Z7”.

I did not expect my real-time quote provider to change year codes for contracts in 2018 and beyond. No longer is it one digit for year —8 in this case. Now it requires two digits. So the December 2018 gold contract is “GC Z18”. Even now that I have looked, I do not find any announcement of this change. I am not even sure if it is an official COMEX change, or just a quirk of one quote provider.

 

Data provider uncertainties

 

This error was compounded because my software was not programmed to notify me of a problem. In software, the only thing worse than a failure in a system that is used in production is a silent failure that goes unnoticed, and hence goes uncorrected. This failure went unnoticed.

 

System Upgrade

Before I get to the impact, I want to discuss how we will make sure this does not happen again.

My team and I have been working hard on a new website, and the centerpiece will be our ongoing data science work in the precious metals markets. We will publish about 45 graphs, with daily updates. Obviously, this is driven by a much more sophisticated software system than my humble application.

The new software is developed by one of the best coders in the world (not me, I’m rusty after not coding full-time in almost 15 years). Rudy Mathieu worked for my last company, a software company called DiamondWare.

Rudy has built a hardened, enterprise-grade software system (now undergoing extensive testing), and when it encounters an error, it does not fail silently. It is constantly checking the status of all key components, and has a dashboard so we can monitor how the software and the server running it are doing.

It emails us if anything goes wrong. It will instantly detect problems, such as a change in the year code or even the Spanish Inquisition, which nobody expects.

 

Nobody expects the Spanish Inquisition!

 

For years, I have been publishing a unique view into the markets. Our new site takes it a thousand times further. I expect that it will become an essential tool for anyone who uses or trades gold. We need to ensure it is as reliable as clockwork.

I promise to make it so.

 

There are ways and means to keep coders line and boost their productivity

 

Data Error Impact

Back to the question: what was the net effect? My software was not able to calculate a basis for gold or silver contracts maturing in 2018 or beyond. However, my fundamental price model relies on them. Its accuracy began to suffer starting around last August.

This error continued to grow in magnitude. As of last week’s Report, the fundamental price of gold was overstated by about $175, and silver’s fundamental price by $2.30.

The correct fundamental prices as of Friday March 31 were about $1,260 and $16.70.

Interestingly — and this is important — the fundamental gold-silver ratio was robust to this error. The value stated in last week’s report, 75.75, was almost perfect. It was off from the revised estimated fundamentals by 0.2.

I say revised and estimated, because I went back over the time period where I have incomplete data and derived what I need. The result is good enough for horseshoes and hand grenades, as we say in America (but it has higher uncertainty).

There is a bigger lesson here. Monetary Metals focuses on the ratio (which we trade in our fund), because it is less error-prone, more accurate, and less risky than trading either metal against the dollar.

The bottom line is that on March 31, the corrected fundamental gold  price was above the market price, though not nearly so far above as I had reported it. It was about 1.4% over the market price (I reported 15.6% last week).

As an aside, my friend Pater Tenebrarum at the Acting Man blog wrote about the disparity between the fundamental drivers that he monitors, and the fundamentals I reported. He is right in thinking that demand for physical is not going ballistic yet.

Though as you will see in the graphs below, the fundamental price has indeed been rising since mid to late December (as I have been correctly reporting), from a low of around $1,115 to $1261 at the end of March.

The correct silver fundamental price is below the market price. My commentary actually stands up pretty well, in light of the correct data. Even while I erroneously reported a silver fundamental price running up to about $19, I have not been enthusiastic about silver. I haven’t “trusted” it enough to encourage a big silver trade, nor called for a major price move. I think there were two reasons.

One, obviously, the nearer-term contracts which I monitor are accurate. They did not show the kind of moves I would expect to see if the market for physical metal was getting so tight. The error only occurred for contracts in 2018. Two, the fundamental gold-silver ratio was correctly calling for a higher market ratio.

Below, I include graphs of the fundamental prices for both metals. The correct values will be overlaid with the ones I have been calculating. So you can see where it went off the rails, and by how far it deviated.

There is one last thing, which I am reluctant to discuss now, before we are ready to launch. Yet it is germane.

Monetary Metals has licensed market data from Thomson Reuters. When the new charts go live, they will be based solely on this data. This data is of better quality than the data from the real-time quote provider I have been using.

And we have developed some very sophisticated algorithms that allows us to extract the maximum signal with the least noise, far superior to what my little app does with the data from my current provider.

The new basis and fundamental prices will not line up perfectly with the old data series. One reason is that the bid-ask spread is tighter. By the nature of the math to calculate the basis, a tighter spread means a higher basis and higher co-basis.

That said, I am confident of two things. One, the new data is more accurate. And two, the old data set has served well in showing the big picture (notwithstanding the error I corrected this week).

 

Fundamental Developments

We will look at the only true picture of supply and demand in the gold and silver markets. But first, the price and ratio charts.

 

Prices of gold and silver – click to enlarge.

 

Next, this is a graph of the gold price measured in silver, otherwise known as the gold-silver ratio. Last week, we asked if the downward-moving gold-silver ratio had hit a line of support. It seems it did, as it moved up sharply on Friday.

 

Gold-silver ratio – click to enlarge.

 

For each metal, we will look at a graph of the basis and co-basis overlaid with the price of the dollar in terms of the respective metal. It will make it easier to provide brief commentary. The dollar will be represented in green, the basis in blue and co-basis in red.

 

Here is the gold graph.

 

Gold basis and co-basis and the dollar price – click to enlarge.

 

Not much change in the scarcity of gold (i.e. the red line, the cobasis) while the price moved up slightly. Our calculated fundamental price is up $30, to about $1,290.

Let’s take a look at two graphs. Both show enough time to see where the error began to creep in, and where it ends. They are May 3, 2016 through March 31, 2017.

The first is the continuous gold basis, with the erroneous line overlaid with the corrected line. As you can see, the erroneous basis was lower (indicating, falsely, lower abundance) and the erroneous co-basis was higher (indicated greater scarcity).

 

Reported and corrected gold basis and co-basis

 

The second is the market price of gold, overlaid with the erroneous and corrected fundamental prices.

The Reported and Corrected Gold Fundamental Prices

 

Gold: market price, reported fundamental price and corrected fundamental price – click to enlarge.

 

The erroneous one takes off for the stars. The corrected value is much closer to the market price, though a bit above.

Now let’s look at silver.

 

Silver basis and co-basis and the dollar price – click to enlarge.

 

We switched from the May to the July contract, as the May contract is in the process of being rolled (where traders must close positions in May and if they want to keep their positions, open a July or farther-out contract).

There is a small decrease in the basis and increase in co-basis, along with a falling price this week. And our fundamental price is up 14 cents, to just under $16.85. Yes, alas, that is more than a buck under the market.

Here are the same extra two graphs for silver.

 

Reported and corrected silver basis and co-basis – click to enlarge.

 

Silver: market price, reported fundamental price, corrected fundamental price – click to enlarge.

 

You can see here that there are two salient features. One, the fundamental price has been rising for about a month longer than that of gold, although from a much more volatile bottom ($12.39). Two, the fundamental price is way below the market price.

 

© 2017 Monetary Metals

 

Image captions by PT

 

Dr. Keith Weiner is the president of the Gold Standard Institute USA, and CEO of Monetary Metals. Keith is a leading authority in the areas of gold, money, and credit and has made important contributions to the development of trading techniques founded upon the analysis of bid-ask spreads. Keith is a sought after speaker and regularly writes on economics. He is an Objectivist, and has his PhD from the New Austrian School of Economics. He lives with his wife near Phoenix, Arizona.

 

 

 

Emigrate While You Can... Learn More

 


 

 
 

Dear Readers!

You may have noticed that our so-called “semiannual” funding drive, which started sometime in the summer if memory serves, has seamlessly segued into the winter. In fact, the year is almost over! We assure you this is not merely evidence of our chutzpa; rather, it is indicative of the fact that ad income still needs to be supplemented in order to support upkeep of the site. Naturally, the traditional benefits that can be spontaneously triggered by donations to this site remain operative regardless of the season - ranging from a boost to general well-being/happiness (inter alia featuring improved sleep & appetite), children including you in their songs, up to the likely allotment of privileges in the afterlife, etc., etc., but the Christmas season is probably an especially propitious time to cross our palms with silver. A special thank you to all readers who have already chipped in, your generosity is greatly appreciated. Regardless of that, we are honored by everybody's readership and hope we have managed to add a little value to your life.

   

Bitcoin address: 12vB2LeWQNjWh59tyfWw23ySqJ9kTfJifA

   
 

5 Responses to “Mea Culpa – Precious Metals Supply and Demand”

  • All-Your-Gold-Are-Mine:

    Mr. Weiner, NO apology necessary… as readers, we’re not doing any of the work, just benefiting from your insights. Please excuse the arrogance of people thinking that you need an “accepted” apology. Of course you do not. And thank you for taking the time to explain in the first person. Personally, I like your writing style in the first person. Again, thanks for your hard work explaining your understanding of how the price of the precious metals, Gold and Silver, are derived.

  • Skew Views:

    When you say, ‘this new site goes a thousand times further’ is that number rounded up or down ? Because it seems hard to believe it will go exactly one thousand times further.
    You may want to double check that data as well. Maybe you meant to use the number 1000 instead.

    • All-Your-Gold-Are-Mine:

      Not funny “Skew” nor witty… why don’t YOU spend hours informing us of YOUR views, before acting like a “smart-*ss”?!

  • Riechers:

    Thank you for your honest disclosure, we all make mistakes, few take ownership. As a regular reader I had noticed the widening margin so appreciate the clarification. I can only imagine how many undisclosed errors live in the thousands of algos that control today’s markets that will never be revealed until it is too late.

  • Hans:

    Mr weiner, apology accepted!

    To ensure more reliable information, we are switching
    your servers to that of HRC; which will guaranty data flow
    from official sources.

Your comment:

You must be logged in to post a comment.

Most read in the last 20 days:

  • The Capital Structure as a Mirror of the Bubble Era
      Effects of Monetary Pumping on the Real World As long time readers know, we are looking at the economy through the lens of Austrian capital and monetary theory (see here for a backgrounder on capital theory and the production structure). In a nutshell: Monetary pumping falsifies interest rate signals by pushing gross market rates below the rate that reflects society-wide time preferences; this distorts relative prices in the economy and sets a boom into motion – which is characterized by...
  • How to Get Ahead in Today’s Economy
      “Literally On Fire” This week brought forward more evidence that we are living in a fabricated world. The popular story-line presents a world of pure awesomeness. The common experience, however,  falls grossly short.   There are many degrees of awesomeness, up to total awesomeness – which is where we are these days, in the age of total awesomeness, just a short skip away from the Nirvana era. What is Nirvana, you may wonder? We only know for sure that Nirvana is what...
  • Full Faith and Credit in Counterfeit Money
      A Useful Public Service There are nooks and corners in every city where talk is cheap and scandal is honorable.  The Alley, in Downtown Los Angeles, is a magical place where shrewd entrepreneurs, shameless salesmen, and downright hucksters coexist in symbiotic disharmony.  Fakes, fugazis, and knock-offs galore, pack the roll-up storefronts with sparkle and shimmer.   The Alley in LA – in places such as this, consumers are as a rule well served by applying a little bit of...
  • Gold and Gold Stocks – Conundrum Alert
      Moribund Meandering Earlier this week, the USD gold price was pushed rather unceremoniously off its perch above the $1300 level, where it had been comfortably ensconced all year after its usual seasonal rally around the turn of the year. For a while it seemed as though the $1,300 level may actually hold, but persistent US dollar strength nixed that idea. Previously many observers (too many?) expected gold to finally break out from its lengthy consolidation pattern, but evidently the...
  • US Money Supply Growth Jumps in March , Bank Credit Growth Stalls
      A Movie We Have Seen Before – Repatriation Effect? There was a sizable increase in the year-on-year growth rate of the true US money supply TMS-2 between February and March. Note that you would not notice this when looking at the official broad monetary aggregate M2, because the component of TMS-2 responsible for the jump is not included in M2. Let us begin by looking at a chart of the TMS-2 growth rate and its 12-month moving average.   The y/y growth rate of TMS-2...
  • Fear and Longing - Precious Metals Supply and Demand
      Waiting for Permanent Backwardation  The price of gold dropped 9 bucks, while that of silver rose 3 cents. Readers often ask us if permanent backwardation (when gold withdraws its bid on the dollar) is still coming. We say it is certain (unless we can avert it by offering interest on gold at large scale). They ask is it imminent, and we think this is with a mixture of fear and longing for a higher gold price.   Lettuce hope this treasure is not cursed... but it probably is....
  • Scorn and Reverence - Precious Metals Supply and Demand
      Shill Alarm One well-known commentator this week opined about the US health care industry:   “...the system is designed the churn and burn... to push people through the clinics as quickly as possible. The standard of care now is to prescribe some medication (usually antibiotics) and send people on their way without taking the time to conduct a comprehensive examination.”   From the annals of modern health care... [PT]   Nope. That is not the standard...
  • Global Turn-of-the-Month Effect – An Update
      In Other Global Markets the “Turn-of-the-Month” Effect Generates Even Bigger Returns than in the US The “turn-of-the-month” effect is one of the most fascinating stock market phenomena. It describes the fact that price gains primarily tend to occur around the turn of the month. By contrast, the rest of the time around the middle of the month is typically far less profitable for investors.   Good vs. bad seasonal timing...   [PT]   The effect has been studied...
  • Tales from “The Master of Disaster”
      Tightening Credit Markets Daylight extends a little further into the evening with each passing day.  Moods ease.  Contentment rises.  These are some of the many delights the northern hemisphere has to offer this time of year. As summer approaches, and dispositions loosen, something less amiable is happening.  Credit markets are tightening.  The yield on the 10-Year Treasury note has exceeded 3.12 percent.   A change in pace: yields are actually going somewhere. There is...
  • Is Political Decentralization the Only Hope for Western Civilization?
      Voting with their Feet A couple of recent articles have once more made the case, at least implicitly, for political decentralization as the only viable path which will begin to solve the seemingly insurmountable political, economic, and social crises which the Western world now faces.   Fracture lines – tax and regulatory competition allows people to “vote with their feet” - and they certainly do. [PT]   In the last few months, over 3,000 millionaires have...
  • Why the Fundamental Gold Price Rose - Precious Metals Supply and Demand
      Gold Lending and Arbitrage There was no rise in the purchasing power of gold this week. The price of gold fell $22, and that of silver $0.19. One question that comes up is why is the fundamental price so far above the market price? Starting in January, the fundamental price began to move up sharply, and the move sustained through the end of April.   1-month LIBOR (London Interbank Offered Rate – the rate at which banks lend euro-dollars to each other). LIBOR and GOFO...
  • “Sell In May And Go Away” - A Reminder: In 9 Out Of 11 Countries It Makes Sense To Do So
      A Truism that is Demonstrably True Most people are probably aware of the adage “sell in May and go away”. This popular seasonal Wall Street truism implies that the market's performance is far worse in the six summer months than in the six winter months. Numerous studies have been undertaken in this context particularly with respect to US stock markets, and they  confirm that the stock market on average exhibits relative weakness in the summer.   Look at the part we...

Support Acting Man

Item Guides

Top10BestPro
j9TJzzN

The Review Insider

Dog Blow

Austrian Theory and Investment

Archive

350x200

THE GOLD CARTEL: Government Intervention on Gold, the Mega Bubble in Paper and What This Means for Your Future

Realtime Charts

 

Gold in USD:

[Most Recent Quotes from www.kitco.com]

 


 

Gold in EUR:

[Most Recent Quotes from www.kitco.com]

 


 

Silver in USD:

[Most Recent Quotes from www.kitco.com]

 


 

Platinum in USD:

[Most Recent Quotes from www.kitco.com]

 


 

USD - Index:

[Most Recent USD from www.kitco.com]

 

Mish Talk

 
Buy Silver Now!
 
Buy Gold Now!
 

Oilprice.com

Diary of a Rogue Economist