Stock Market Tools (11) - Download Amibroker Historical Data 10 x Faster or More...

Generally, it takes only a couple of minutes to download the current data for the Amibroker database with Amiquote.  But, if one were to miss a day without doing the current update, he would need to do a historical update to back fill the database.  Which can last a couple of hours if the database is huge, let say a database with 5,000 tickers or more.

This video here presents is a trick to speed up the download up to 10 times faster or even more, making the complete back-fill of historical data within  a tea break time.

(Added on 16 Jan 2014 morning - you may save your time in watching the video by reading the comment by Anonymous, which is attached below.)

(Added on 16 Jan 2014 afternoon - you may still watch the video and combine it with the method as Anonymous suggested, it can literally speed up the Historical Download up to 10 x 10 => 100 times!!!  Just tested working.  The CPU multi-task does not depends on the number of core, but i7 Chips does help to make the operation smoother.)


.
If it is not clear, you might want to go to YouTube Link to watch it in HD Mode:  Link

By the way, if you do come across other easier or faster method, please comment/share. :-)


Bless You
KH Tang


HOME

Great! Just posted one day, and some one gives a better solution!
--------------------------------------------------------------------------------------------------
Added on 16 Jan 2014
Anonymous said...
I do it in the following way to achieve similar result (I guess):

go to amiQuote->Settings-> change the "Number of simultaneous downloads" to say 8 since i7 provides 8 threads simultaneously.
--------------------------------------------------------------------------------------------------

Thanks "Anonymous"!

I was not aware of this feature and it definitely works better than my old manual way of doing.  And, it works for "current update" too!

It just speed up all download up to 10x when I set the "Number of simultaneous downloads" to 10.

--------------------------------------------------------------------------------------------------

抛砖引玉

Throw a Brick to Attract a Jade

抛砖引玉，出自《传灯录》。相传唐代诗人常建，听说赵嘏要去游览苏州的灵岩寺。为了请赵嘏作诗，常建先在庙壁上题写了两句，赵嘏见到后，立刻提笔续写了两句，而且比前两句写得好。后来文人称常建的这种作法为“抛砖引玉”。

In Tang dynasty, there was a poet JianChang. Once he heard that another poet GuZhao was going to visit LinYan Temple in ShuZhou province, and wanted to get GuZhao to write a poem.  So, JianChang went to the Temple first and wrote half a poem on the wall.  When GuZhao arrived, by his nature, he immediately took a pen and finished the poem.  And, the second half is much better!  And, people call this “Throw a brick to attract a jade.”

-------------------------------------- Added on 16 Jan 2014 afternoon --------------------------------
I  combined both methods mentioned above, and I was able to download more than 5000 historical data (1 week back-fill) within 5 minutes!

Cheers!
KH Tang



------------------------------------------------- Added on 16 Jan 2014 Evening -----------------------------------------------------
Amibroker Said:

Be careful with that. While using 10 simultaneous downloads is relatively safe,
using more than that will sooner or later result in Yahoo blocking your IP for abuse.

In the past Yahoo already blocked people who were using too many simultaneous downloads
and it was one of the reasons why default in AmiQuote has been changed from 10 to 1 (fail-safe).