OANDA DATA

Sources of data to import to the program
Message
Author
Yukoner
Posts: 6
Joined: Fri Jan 12, 2007 7:19 pm
Contact:

OANDA DATA

#1 Postby Yukoner » Fri Jan 12, 2007 7:27 pm

Hi guys,
I received tick data from Oanda that gives me eur/usd from 2004 onwards. It appears to be in .txt format. Now when i try to import the data to forextester i run into errors. The file is huge when unzipped, but as i understand it is very, very accurate.

I'm not a programmer, so any help would be appreciated... but would need to be step by step please.

For the record I really like Forextester. It has helped me immensely in my testing, and because of that my real time trading results have also improved. You visually test 500 trades, and taking one for real isn't that big of a deal.

Thanks,
Jeff Wagner

User avatar
Terranin
Site Admin
Posts: 833
Joined: Sat Oct 21, 2006 4:39 pm

Re: OANDA DATA

#2 Postby Terranin » Fri Jan 12, 2007 9:31 pm

Yukoner wrote:Hi guys,
I received tick data from Oanda that gives me eur/usd from 2004 onwards. It appears to be in .txt format. Now when i try to import the data to forextester i run into errors. The file is huge when unzipped, but as i understand it is very, very accurate.

I'm not a programmer, so any help would be appreciated... but would need to be step by step please.

For the record I really like Forextester. It has helped me immensely in my testing, and because of that my real time trading results have also improved. You visually test 500 trades, and taking one for real isn't that big of a deal.

Thanks,
Jeff Wagner


Can you send us this file or part of it that we can test error?
Hasta la vista
Mike

Yukoner
Posts: 6
Joined: Fri Jan 12, 2007 7:19 pm
Contact:

You should have it now.

#3 Postby Yukoner » Mon Jan 15, 2007 11:16 am

Sent it over the weekend.

Jeff

Trillion
Posts: 24
Joined: Sat Jan 20, 2007 8:17 pm

OANDA tick data

#4 Postby Trillion » Sat Jan 20, 2007 8:40 pm

I use OANDA data too. You can specify how you would like the tick data file delimited (txt, csv, pipe-delimited or XML). I have mine in .csv format but OANDA have not used a comma to separate the date column from the tick data. Thus, each line of data looks like this (Date, Time, Tick Value 1, Tick Value 2):-

01/01/04 07:43:00,1.258700,1.259700
01/01/04 07:47:52,1.258500,1.259500
01/01/04 17:46:14,1.258600,1.259600
01/01/04 17:56:08,1.258500,1.259500
01/01/04 17:56:15,1.258500,1.259500
01/01/04 17:56:28,1.258500,1.259500
01/01/04 17:56:30,1.258500,1.259500
01/01/04 17:56:40,1.258500,1.259500
01/01/04 17:57:35,1.258500,1.259500

Using the <import history> function in Forex Tester, how is it possible to import this data file?

User avatar
Terranin
Site Admin
Posts: 833
Joined: Sat Oct 21, 2006 4:39 pm

Re: OANDA tick data

#5 Postby Terranin » Mon Jan 22, 2007 8:17 pm

Trillion wrote:I use OANDA data too. You can specify how you would like the tick data file delimited (txt, csv, pipe-delimited or XML). I have mine in .csv format but OANDA have not used a comma to separate the date column from the tick data. Thus, each line of data looks like this (Date, Time, Tick Value 1, Tick Value 2):-

01/01/04 07:43:00,1.258700,1.259700
01/01/04 07:47:52,1.258500,1.259500
01/01/04 17:46:14,1.258600,1.259600
01/01/04 17:56:08,1.258500,1.259500
01/01/04 17:56:15,1.258500,1.259500
01/01/04 17:56:28,1.258500,1.259500
01/01/04 17:56:30,1.258500,1.259500
01/01/04 17:56:40,1.258500,1.259500
01/01/04 17:57:35,1.258500,1.259500

Using the <import history> function in Forex Tester, how is it possible to import this data file?


You can not load this data directly to ForexTester. It must be converted first into 1 minute data.
Hasta la vista

Mike

Yukoner
Posts: 6
Joined: Fri Jan 12, 2007 7:19 pm
Contact:

Converting Oanda data to 1min bars

#6 Postby Yukoner » Mon Jan 22, 2007 11:18 pm

Mike,
Could you give us the steps on how to do that, or is there a link on the site how to do that.
Thanks,
Jeff

Trillion
Posts: 24
Joined: Sat Jan 20, 2007 8:17 pm

#7 Postby Trillion » Tue Jan 23, 2007 5:27 pm

Is 1 minute data the smallest granularity that Forex Tester can handle?

Trillion
Posts: 24
Joined: Sat Jan 20, 2007 8:17 pm

Re: OANDA tick data

#8 Postby Trillion » Tue Jan 23, 2007 5:30 pm

Terranin wrote:
Trillion wrote:I use OANDA data too. You can specify how you would like the tick data file delimited (txt, csv, pipe-delimited or XML). I have mine in .csv format but OANDA have not used a comma to separate the date column from the tick data. Thus, each line of data looks like this (Date, Time, Tick Value 1, Tick Value 2):-

01/01/04 07:43:00,1.258700,1.259700
01/01/04 07:47:52,1.258500,1.259500
01/01/04 17:46:14,1.258600,1.259600
01/01/04 17:56:08,1.258500,1.259500
01/01/04 17:56:15,1.258500,1.259500
01/01/04 17:56:28,1.258500,1.259500
01/01/04 17:56:30,1.258500,1.259500
01/01/04 17:56:40,1.258500,1.259500
01/01/04 17:57:35,1.258500,1.259500

Using the <import history> function in Forex Tester, how is it possible to import this data file?


You can not load this data directly to ForexTester. It must be converted first into 1 minute data.


How do you suggest that I go about this?

User avatar
Terranin
Site Admin
Posts: 833
Joined: Sat Oct 21, 2006 4:39 pm

Re: Converting Oanda data to 1min bars

#9 Postby Terranin » Tue Jan 23, 2007 5:39 pm

Yukoner wrote:Mike,
Could you give us the steps on how to do that, or is there a link on the site how to do that.
Thanks,
Jeff


There is no link. Somebody should program a converter, possible me when I have some free time.
Hasta la vista

Mike

User avatar
Terranin
Site Admin
Posts: 833
Joined: Sat Oct 21, 2006 4:39 pm

#10 Postby Terranin » Tue Jan 23, 2007 5:41 pm

Trillion wrote:Is 1 minute data the smallest granularity that Forex Tester can handle?


This is the smallest granularity to import, but when you generate ticks they will be created inside 1 minute bars by 3 different methods you can choose.
Hasta la vista

Mike

Trillion
Posts: 24
Joined: Sat Jan 20, 2007 8:17 pm

#11 Postby Trillion » Tue Jan 23, 2007 6:40 pm

Given that Forex Tester limits price data to 1 minute granularity, how can the software accurately generate exact tick prices as seen by the market?

Well, after reading the help file and specifically the sub section relating to tick data, 'Generate tick data', I see that Forex Tester does not accurately draw tick data at all! Instead, the methods described only mimic the OHLC price, generating ticks either point by point, as OHLC price values or randomly! This is a real pity ...

Do you not think it would be better to cater for proper tick data in the first instance?

Trillion
Posts: 24
Joined: Sat Jan 20, 2007 8:17 pm

#12 Postby Trillion » Tue Jan 23, 2007 6:48 pm

Don't get me wrong, I'm not saying that the 3 methods don't have a use just that given pure tick data, Forex Tester ought to be able to backtest strategies from each tick in the file and also be able to build complete bars of any time frame from the data. Do you think this is a good idea?

User avatar
Terranin
Site Admin
Posts: 833
Joined: Sat Oct 21, 2006 4:39 pm

#13 Postby Terranin » Tue Jan 23, 2007 7:05 pm

Trillion wrote:Don't get me wrong, I'm not saying that the 3 methods don't have a use just that given pure tick data, Forex Tester ought to be able to backtest strategies from each tick in the file and also be able to build complete bars of any time frame from the data. Do you think this is a good idea?


For the most testing purposes it is more than enough. Most traders use 1 hour timeframe at least, only pipsers try to work with every tick to get 1-5 pips in every transaction but my experience tells me that brokers fight with those people, force them to use not market but pending orders. Make delays in order execution on the fast price movement or use requotes. So I really do not see any value to use exact ticks data, 1 minute is enough.

But if you want to test as precisely, you can replace ticks arrays generated by program at "\data\Ticks" folder with your own. Then it will be exactly the same ticks. Still data converter is required.
Hasta la vista

Mike

Trillion
Posts: 24
Joined: Sat Jan 20, 2007 8:17 pm

#14 Postby Trillion » Tue Jan 23, 2007 7:37 pm

Let say I want to use a strategy which involves buying or selling above or below the previous bar's OHLC. In real-time, the price moves up and I go long above the high of the last bar and set my stop and reverse accordingly. Using the methods employed by Forex Tester to generate ticks, on playback I may well find myself short rather than long which would not accurately portray the situation as it developed, leading to inaccurate backtesting results.

However, if Forex Tester can work accurately with tick data which is substituted then, wonderful! One thing though ... isn't the actual tick progression lost in conversion?

User avatar
Terranin
Site Admin
Posts: 833
Joined: Sat Oct 21, 2006 4:39 pm

#15 Postby Terranin » Tue Jan 23, 2007 8:22 pm

Trillion wrote:Let say I want to use a strategy which involves buying or selling above or below the previous bar's OHLC. In real-time, the price moves up and I go long above the high of the last bar and set my stop and reverse accordingly. Using the methods employed by Forex Tester to generate ticks, on playback I may well find myself short rather than long which would not accurately portray the situation as it developed, leading to inaccurate backtesting results.

However, if Forex Tester can work accurately with tick data which is substituted then, wonderful! One thing though ... isn't the actual tick progression lost in conversion?


Do you play on 1 minute bars? Only in this case that could happen. (possible) Because to generate ticks Forex Tester use 1 minute data, so in 1 hour bar it will be 60x4 ticks generated with OHLC method. Or much more if you generate ticks point by point. And mostly 1 minute bar is small < 10 points so difference between 2 ticks is about 3-4 points.

What does it mean "actual progression"? You will get absolutely the same ticks you published before. But I have to write converter first.

I wrote this software for myself first, to test automated systems on it, and I am fully satisfied with the quality of results. After testing my systems on Forex Tester I translated them to Meta Trader and tested on demo account, and I got the same results.
Hasta la vista

Mike

Trillion
Posts: 24
Joined: Sat Jan 20, 2007 8:17 pm

#16 Postby Trillion » Tue Jan 23, 2007 8:36 pm

I don't trade 1m bars but I may be onto a strategy that is effective on such a tiny time frame.

The actual progression is one of 3 ticks up, 8 ticks down, 2 up, 3 more up, 2 down ... until the bar is complete.

thanks for bearing with me on this, I just need to be sure that I can use OANDA's tick data with Forex Tester.

capsmart
Posts: 12
Joined: Sun Jan 14, 2007 3:11 pm
Location: GREECE

I agree with Mike

#17 Postby capsmart » Wed Jan 24, 2007 4:27 am

Hi
Tick data is random to me. It can go up or down without any reason. So basing a strategy on history ticks or on random ticks to me is the same.
Good luck with your system because you will need it.
Regards

P.S.
Mike any news concerning the PSAR and the 30 min interval?

Trillion
Posts: 24
Joined: Sat Jan 20, 2007 8:17 pm

Re: I agree with Mike

#18 Postby Trillion » Wed Jan 24, 2007 6:38 am

capsmart wrote:Hi
Tick data is random to me. It can go up or down without any reason.


Hmmm ... You must be new to trading so I'll explain to you what is happening when you see a tick. I'll start with the exact definition of a tick which is, the minimum fluctuation in a price, either up or down. When you see the price tick that tells you that the price is moving and when price moves that means there is activity in the market place. It's that activity that I watch. Now, down to the level of market ticks, when grouped together over a period it can reveal price levels that offer intraday support and resistance and it's these levels that many traders accept as intraday pivots - no matter what the time frame used for trading - and from these levels strategies can be found. So you see that not only is there 'reason' but also that ticks can offer insight into price action not immediately obvious on larger time frames.

Perhaps you don't trade intraday or perhaps support and resistance levels mean nothing to you? Perhaps you don't trade at all? Whatever that case, just because you see tick data as random doesn't mean that others can't see the value in them.

I hope this insight has been some use to you, if not least that you understand that ticks are not random.

Best wishes ...

capsmart
Posts: 12
Joined: Sun Jan 14, 2007 3:11 pm
Location: GREECE

#19 Postby capsmart » Wed Jan 24, 2007 10:35 am

You have your point and I can understand it.
I am mostly a stock trader on daily chart and a forex intraday trader using 30 min charts.
On the other hand if the lowest 30 min bar of the day is 30 pips long I will consider the lowest point as an eventual support and you might consider the middle of the bar as support since there is where you noticed the move.
But do you remember exactly where was the move and the speed of it?
And you are watching the screen all day on a tick by tick chart?
And can you react to a pip level for entries and exits?
I am too old for that.
But of course everyone has his own system.
The only inconvience would be the size of the file to download otherwise if you can trade on a tick chart that is fine for me and I am sure profitable for you.
Regards

User avatar
Terranin
Site Admin
Posts: 833
Joined: Sat Oct 21, 2006 4:39 pm

Data converters

#20 Postby Terranin » Mon Jan 29, 2007 10:25 pm

Here it is 2 data converters from OANDA tick data:

DataConvToCsv.exe - this will convert tick data to *.csv file that Forex Tester is able to import.

to convert type in command line:
DataConvToCsv.exe <Oanda tick file.txt>

you will have resulting *.csv file that could be imported to Forex Tester as usual.


DataConv.exe - this will convert tick data to Forex Tester tick file, so you will get exactly the same ticks during testing.

to convert type in command line:
DataConv.exe <Oanda tick file.txt>

resulting file will have the same name + "_xx.dat"

To test this data:
1. Generate ticks for this pair(s), even 1 day of data, it is
necessary for FT to fill currency table.
2. Replace file(s) at <FT>\data\Ticks\ with converted files (rename
EURUSD_xx.dat to EURUSD.dat for example).
3. Press connect and enjoy. :)
Attachments
DataConv.zip
(53.23 KiB) Downloaded 2194 times
DataConvToCsv.zip
(54.55 KiB) Downloaded 2082 times
Hasta la vista

Mike

Yukoner
Posts: 6
Joined: Fri Jan 12, 2007 7:19 pm
Contact:

Trying to make the convertors work

#21 Postby Yukoner » Tue Mar 20, 2007 3:12 pm

Hi Mike,

Thanks for the convertors. I just can't seem to get either one to work. I believe the fault lies on my side. For some reason i can't even get it to load....

If its not too much trouble, could you give a step by step instruction on how to get the convertors up and running? Pretending i know nothing... which is pretty close to the truth when it comes to programming.

Thanks,
Yukoner :?

User avatar
Terranin
Site Admin
Posts: 833
Joined: Sat Oct 21, 2006 4:39 pm

Re: Trying to make the convertors work

#22 Postby Terranin » Tue Mar 20, 2007 7:46 pm

Yukoner wrote:Hi Mike,

Thanks for the convertors. I just can't seem to get either one to work. I believe the fault lies on my side. For some reason i can't even get it to load....

If its not too much trouble, could you give a step by step instruction on how to get the convertors up and running? Pretending i know nothing... which is pretty close to the truth when it comes to programming.

Thanks,
Yukoner :?


Do you have some file manager like windows commander where you can type commands? To use for example DataConvToCsv.exe converter you should download oanda tick file to the same directory and then in file manager type "DataConvToCsv.exe <Oanda tick file.txt>" where <Oanda tick file.txt> equals to the file name of the oanda tick file that you downloaded.

DataConvToCsv.exe somefile.txt (for example)

and it will create "somefile.csv" file at the same directory.
Hasta la vista

Mike

Matter13
Posts: 4
Joined: Sat Nov 11, 2006 3:18 pm

#23 Postby Matter13 » Tue May 22, 2007 1:17 pm

Hi Mike,

Thx for converters. I have the following problems/questions:

Does converters convert .txt (not comma seperated) or .csv(comma seperated) files from Oanda?

I have Oanda tick data which is not seperated by comma but 'space'.
When i convert via Datacontocsv it goes thru ok but i get the file with no data in it.

If i want to convert directly to tick via Dataconv, does data have to be prior converted with Dataconvtocsv?

Thx, Mato

Matter13
Posts: 4
Joined: Sat Nov 11, 2006 3:18 pm

#24 Postby Matter13 » Tue May 22, 2007 5:06 pm

Just figured it out...
Had to convert Oanda tick data i had (seperated by 'space), to coma seperated so that it looks exactly like the one posted in previous post.

It works nicely now!

Kaizen
Posts: 1
Joined: Sun Mar 02, 2008 5:24 am

#25 Postby Kaizen » Sun Mar 02, 2008 5:30 am

After much searching and trying a few solutions to convert Oanda tick data I found free converter at http://www.tickdataconverter.com

The one provided by FT did not quite do what I needed.


Return to “Data sources”

Who is online

Users browsing this forum: No registered users and 19 guests