ESignal data?

[brett0]

Hey,

Has anyone successfully imported ESignal data, either on its own or with QCollector?

An example of a data set from QCollector/Esignal in ASCII:

08/27/2008,09:35,78.17,78.17,78.00,78.00,76400
08/27/2008,09:40,78.00,78.05,77.77,78.05,50500
08/27/2008,09:45,78.05,78.09,78.00,78.08,49200
08/27/2008,09:50,78.05,78.10,77.96,77.96,31400
08/27/2008,09:55,77.98,78.14,77.93,78.07,33600
08/27/2008,10:00,78.06,78.10,77.70,77.92,63100
08/27/2008,10:05,77.93,77.93,77.80,77.80,40600
08/27/2008,10:10,77.80,77.88,77.70,77.82,39700
08/27/2008,10:15,77.86,77.87,77.80,77.80,29600
08/27/2008,10:20,77.80,78.01,77.78,78.01,57400
08/27/2008,10:25,77.98,78.07,77.90,77.96,35200
08/27/2008,10:30,77.92,78.04,77.80,77.94,33900

It looks like it should work fine, I just figured I'd get a second opinion to be sure, since QCollector isn't free (although it is relatively inexpensive).

Thanks in advance for any help or advice.

[FT Support]

Hello,

it should be possible to import such data, you just need to remove "/" and ":" symbols in it.

[brett0]

[quote="FT Support"]Hello,

it should be possible to import such data, you just need to remove "/" and ":" symbols in it.[/quote]

Thanks, I have successfully imported the data and it works perfect in history mode.

However, I am having a strange issue with volume (using the provided volume indicator, not the "Show Volume" chart option) when I use the data in testing mode. I am choosing the option to generate ticks with real volume, but the volume for some of the periods comes up blank.

I have attached an example 5 minute chart in both history mode and testing mode (it shows up fine in history mode), and I have pasted the CSV data from for the 5 minutes before and after this period:

20120104,095000,1.5602,1.5603,1.5595,1.5599,1480
20120104,095100,1.5599,1.5604,1.5597,1.5601,951
20120104,095200,1.56,1.5603,1.5598,1.5602,825
20120104,095300,1.5601,1.5605,1.5598,1.5603,1000
20120104,095400,1.5603,1.5605,1.5599,1.5603,929
20120104,095500,1.5602,1.5606,1.56,1.5604,1078
20120104,095600,1.5603,1.5606,1.5601,1.5604,1024
20120104,095700,1.5604,1.5607,1.56,1.5604,1181
20120104,095800,1.5604,1.561,1.5602,1.5608,1129
20120104,095900,1.5606,1.561,1.5602,1.5607,1880
20120104,100000,1.5609,1.5613,1.5605,1.5609,1533

As you can see from the chart, it's not just this period, I'm just providing this period as an example. It happens seemingly at random.

Am I doing something wrong when generating the ticks? Any help would be appreciated.

[brett0]

Also, here is a screenshot taken with the "Show Volume" checked alongside the volume indicator. Even the volume on the chart is showing blank, so it doesn't look like its a bug with the volume indicator itself.

[FT Support]

Hello,

Hmm, what do you see in "data" window if you hover the cursor over the candle with empty volume?

[brett0]

[quote="FT Support"]Hello,

Hmm, what do you see in "data" window if you hover the cursor over the candle with empty volume?[/quote]

It says "Volume 20". It should be "6292". I checked a few other of the empty volume bars and it looks like almost all of them have "20" as the volume, with a few of them at 19. Everything else for these candles (open, high, low, close, etc) are 100% accurate.

[FT Support]

Hello,

Lets analyze imported data at these places, what volumes do you have there? can you please post some small amount of data which contains a problem with volume on a chart after the import.

[brett0]

I pasted the data for the imported 1 minute bars that were in the immediate vicinity of the 9:55 bar in post 3 of this thread, but here are the imported 1 minute bars from 9:45 to 10:15 in case the problem is not with the immediate data:

20120104,094500,1.5594,1.5601,1.5593,1.5596,1196
20120104,094600,1.5596,1.5599,1.5593,1.5595,1393
20120104,094700,1.5597,1.5598,1.5593,1.5597,1049
20120104,094800,1.5597,1.5598,1.5591,1.5593,849
20120104,094900,1.5595,1.5602,1.5594,1.5602,1370
20120104,095000,1.5602,1.5603,1.5595,1.5599,1480
20120104,095100,1.5599,1.5604,1.5597,1.5601,951
20120104,095200,1.56,1.5603,1.5598,1.5602,825
20120104,095300,1.5601,1.5605,1.5598,1.5603,1000
20120104,095400,1.5603,1.5605,1.5599,1.5603,929
20120104,095500,1.5602,1.5606,1.56,1.5604,1078
20120104,095600,1.5603,1.5606,1.5601,1.5604,1024
20120104,095700,1.5604,1.5607,1.56,1.5604,1181
20120104,095800,1.5604,1.561,1.5602,1.5608,1129
20120104,095900,1.5606,1.561,1.5602,1.5607,1880
20120104,100000,1.5609,1.5613,1.5605,1.5609,1533
20120104,100100,1.5609,1.5611,1.5605,1.5609,1086
20120104,100200,1.5608,1.5611,1.5602,1.5604,1182
20120104,100300,1.5604,1.561,1.5603,1.5607,1321
20120104,100400,1.5605,1.5608,1.5602,1.5605,964
20120104,100500,1.5605,1.5607,1.5602,1.5605,747
20120104,100600,1.5604,1.5605,1.5595,1.5597,843
20120104,100700,1.5598,1.5602,1.5596,1.5599,1117
20120104,100800,1.5597,1.56,1.5595,1.5598,1000
20120104,100900,1.5597,1.56,1.5593,1.5595,930
20120104,101000,1.5596,1.5599,1.5594,1.5599,881
20120104,101100,1.5599,1.5606,1.5596,1.5605,920
20120104,101200,1.5605,1.561,1.5604,1.5607,871
20120104,101300,1.5605,1.5609,1.5604,1.5608,855
20120104,101400,1.5607,1.5609,1.5602,1.5605,949
20120104,101500,1.5606,1.5615,1.5605,1.5613,1124

Please let me know if I need to provide more data, or if you would like me to upload the entire data file etc.

[brett0]

Okay, here are two screen shots that I think more clearly demonstrate the problem.

One char is the 1 minute time frame in test mode, the other is the 1 minute time frame in history mode.

I have marked the 9:55 spot again on both charts so that it is easy to visually compare the imported data pasted above with the visual representation.

[brett0]

After I posted those 1 minute charts I put the cursor over each candle from 9:45 to 10:15 and looked at the "Data" window to see if anything stood out. Here's what I found:

1) Every candle in "history mode" is 100% accurate
2) Every candle in "testing mode" that has a volume of LESS THAN 1001 is 100% accurate
3) Every candle in "testing mode" that has a volume of MORE THAN 1000 is reported as either 3 or 4

So, it looks like when the ticks are generated, something happens if there are more than 1000 ticks.

The candles that are correct in the test mode screenshot look different than those that are correct in the history mode screenshot because the scale is different, due to the highest volume candles not being the same in both screenshots (since there are no 1001+ volume candles in testing mode).

Here is the exact breakdown of volume that is reported in the "Data" window in "Test Mode" for the 1 minute chart (the correct data is in parenthesis):

09:45 => 4 (1196)
09:46 => 4 (1393)
09:47 => 4 (1049)
09:48 => 849 (CORRECT)
09:49 => 3 (1370)
09:50 => 4 (1480)
09:51 => 951 (CORRECT)
09:52 => 825 (CORRECT)
09:53 => 1000 (CORRECT)
09:54 => 929 (CORRECT)
09:55 => 4 (1078)
09:56 => 4 (1024)
09:57 => 4 (1181)
09:58 => 4 (1129)
09:59 => 4 (1880)
10:00 => 4 (1533)
10:01 => 4 (1086)
10:02 => 4 (1182)
10:03 => 4 (1321)
10:04 => 964 (CORRECT)
10:05 => 747 (CORRECT)
10:06 => 843 (CORRECT)
10:07 => 4 (1117)
10:08 => 1000 (CORRECT)
10:09 => 930 (CORRECT)
10:10 => 881 (CORRECT)
10:11 => 920 (CORRECT)
10:12 => 871 (CORRECT)
10:13 => 855 (CORRECT)
10:14 => 949 (CORRECT)
10:15 => 4 (1124)

[FT Support]

Hello,

Thank you for the details! we'll see if we can fix that.
For now you can use this workaround:

1) Open data in Excel
2) create a formula which will divide all volumes by 10
3) save updated data
4) import updated data and assume that volume 100 is equal to 1000

[brett0]

Thanks! This is actually the same workaround I was considering last night, as the "absolute" amount of volume doesn't matter one bit to me, just the relativity.

In case anyone else is curious about using Esignal data in this manner, here is what I've found:

I pulled up all of the data from March 1st, 2007 to present and I found that only 19 of the 1-minute candles have a volume greater than 5,000 -- 9 of which are greater than 10,000. All of these are from 2008, and almost all of them are from the summer (during the height of the financial panic).

For now I'll either:
(A) max all 1-minute candles at 10,000 (all values lower than 10,000 stay the same, values higher than 10,000 are changed to 10,000) and then divide by 10, or
(B) max all 1-minute candles at 5,000 and then divide by 5 (same idea as A, just to give me a little extra precision), or
(C) just skip the summer of 2008 altogether as it's not really important for the type of trading that I do.. so if I choose to do this I would just divide by 5 (or 10), there's no need to max the candle first -- however, if someone else needs that data for their analysis, (A) or (B) should work just fine

Thanks again. I'll update this thread once I've had a chance to fully test out either (A), (B), or (C) in test mode.