A Benchmark List for the Compact Flash card in NX/NZ

[s360clie]

First of all, I do not have a NX/NZ or a CF Driver. But I'd like to know about the speed of CF on NX/NZ. I think it is useful for the user who hasn't bought their CF card. Althrough there are many benchmarks concerning CF Card's speed on the web, but the result on NX/NZ is much more useful for us.
I prefer to use the software called VFsMark(http://palmgear.com/software/showsoftware.cfm?prodID=34400)
as a standard.
Please list the brand and the capacity of the CF Card and the VFSMark below. Tks.

[timw_de]

Sandisk 512Mb

VFSMark : 135

[timw_de]

Sandisk 16Mb

VFSMark : 154

[timw_de]

PQI 256Mb

VFSMark : 120

[timw_de]

Sandisk 64Mb

VFSMark : 117

[zaks]

Soligor 64 MB

VFSMark : 114

[jtumavit]

Sandisk 256Mb

VFSMark : 128

[n2ifp]

VFS Marks has proven to be bogus and not a true represenation of speed. Not only that, things can change if the developer optimizes the driver even more. I don't want to see anyone misled into thinking one CF is going to be any faster than another at this time. Cards like the SanDisk ULTRA will only perform in a device that is native to that card. I think most will find that the CF will perform a tad better than it's MS counterpart.

[timw_de]

i thought that was weird, i tested th 512Mb Sandisk 3 times and got 3 different results !

[oyakov]

Ritek/Arita 256MB

VFSMark : 170

[lscknight]

Sandisk 128MB
VFSMark : 144

Sandisk 192MB
VFSMark : 162

Lexar x4 64MB
VFSMark : 172

SONY MS 128MB /w MSPRO driver
VFSMark : 143

MSPRO 512MB /w MSPRO driver
VFSMark : 183

[Unregistered]

MS 64MB (143)
Sandisk 192 CF (169)
Sandisk 256 Ultra CF (222)
Sandisk 512 Ultra CF (160)

I would like to know why my 256 blew the 512 out of the water. It is about 35% faster then the 512.

I even tested it with creating a back up. The 512 took 1:30 and the 256 took 50 seconds.

Anyone got any ideas? They were both brand new and the 512 went back to the store. :)

[n2ifp]

That was my whole point, you can drive yourself nuts with VFS testing and results can be all over the map. In order to do it right, you need a controlled enviornment using a proven speed technique. If you have ever seen testing done on desktop PC's, different tests give different results. Where one may lag in one area, it's superior in another.

In any case, have fun :)!!

[CliePet]

My speed measurements (your mileage will vary)

Not too surprising, the VFSMark numbers are not that impressive:

Standard MS stick:
VFSMark = 153 (27% File Write, 153% File Read)

A standard CF memory card:
VFSMark = 124 (73% File Write ??, 120% File Read)

A "30x" faster CF memory card:
VFSMark = 165 (24% File Write, 206% File Read)

In MS Masquerade mode the VFSMark numbers are essentially the same.
Subsequent runs return similar numbers (but can vary by around 5%).

[CliePet]

BTW: I'm not a fan of the way VFSMark tests I/O.

Comparing raw i/o numbers for large read/writes:
(1KB/s = 1000 bytes per second)

CLIE + MS: 624KB/s read, 278KB/s write
CLIE + MS using CF->MS adapter: 819KB/s read, 297KB/s write
[ie. same memory stick used in both tests, but it appears the
CF overhead is less than the built in MS driver
NOTE: the adapter converts an MS stick to a CF interface,
so all the data is still going through the MS serial interface]

CLIE + normal CF: 516KB/s read, 152KB/s write
**** CLIE + faster CF: 924KB/s read, 525KB/s write *****

"normal CF" = older run-of-the-mill Sandisk card
"faster CF" = newer Transcend 256MB "30x" (there are faster ones too)
I have tried other CF cards that fall between these two extremes.

Running similar tests on PocketPC return similar numbers, and in many cases the CLIE (NZ90) speeds are a bit faster, and significantly faster in the "faster CF" case.

-----

So if you are looking at using a CF card as a faster alternative
to a standard memory stick, you should shop around for "faster"
CF cards (24x, 30x, 36x). These typically cost more.

For most practical purposes, the speed variations probably won't
matter much (and a cheap MS stick may be just as effective)

[jimdoc01]

Originally posted by timw_de
Sandisk 64Mb

VFSMark : 117
Hmmm.

My Sandisk 64 gets 173

[sidrum]

My KingSton 512mb CF
1st time VFSMark :205
2nd time VSFMark:207

[Halesy]

Even though n2ifp questions the validity of this test I've done it anyway as an FYI:

512M Muse CF
Masquarading + tricked - VFS mark = 172
Masquarading - VFS mark = 171
No masquarading or trick - VFS mark = 142 (guess this'll be the MS again)
Just Lexmar 128B MS - VFS mark = 147

[frasej]

Sandisk 1GB - 137
(Write: 28%, Read: 223%)

[FireHouse]

My Transcend 256MB (25x) CF = 174

File Create: 189%
File Delete: 111%
File Write: 66%
File Read: 190%
File Seek: 187%
DB Export: 150%
DB Import: 183%
Record Access: 262%
Resource Access: 212%

VFSMark: 174

[n2ifp]

Originally posted by jimdoc01

Hmmm.

My Sandisk 64 gets 173

That's my whole point, VFS marks don't prove anything. Unless done in a controlled enviornment with a multiple of timed real world tests. BTW, in the Clie itself you may not see any real differences. Everyone worries about the fastest card and doesn't even think of reliability. Sort of like having a Ferrari sports car in rush hour traffic.  

[Unregistered]

If it is all bogus why bother posting any of it?

[TriPLeX]

Its not that you have to follow my word, but LISTEN to n2ifp for once. Don't argue about the points your tests gave, but go ahead and post the marks at your will. Just please dont bicker.

(sorry if it sounds like I'm bickering :D )

[=+Pete]

how fast is it?

[berliner]

Viking 256 MB CF: 203
Lexar Media 512 MB MSPro: 194

That being said, my CF is very slow when using Movie Player in masquerade mode as compared to my MSPro. But I think that is more of a driver issue that hopefully Eruware will work out.

[Unregistered]

My SimpleTech 1GB Compactflash Type II
MS Masquerading
File Create: 314%
File Delete: 176%
File Write: 151%
File Read: 248%
File Seek: 207%
DB Export: 316%
DB Import: 196%
Record Access: 270%
Resource Access: 234%

VFSMark: 234

[Unregistered]

My SimpleTech 1GB Compactflash Type II
MS Masquerading
File Create: 314%
File Delete: 176%
File Write: 151%
File Read: 248%
File Seek: 207%
DB Export: 316%
DB Import: 196%
Record Access: 270%
Resource Access: 234%

VFSMark: 234

Sorry, The Unregisterd Guest was me. I was not accepting cookies...

[Halesy]

There's an interesting point, in my post I noted the VFS mark for my CF was better than my MS. I can't however get High Quality movies working on my CF, but I can on my MS. Hope this is resolved by the next release of the driver.

[patrickl]

I really don't understand why n2fip keeps on complaining about VFS Mark. I find the results closely mirror actualy performance. The benchmark itself is the controlled environment. I find the variation in marks between runs to be statistically insignificant (like + or - 1%).

Only problem is that speeds will vary from when cards are filled up and when they are empty and maybe the fact that they include a great part of the CPU overhead. Thus a faster processor will yield higher marks. But then the whole operation is faster so thats again a reflection of factual speed increase.

I do think that the VFS mark in itself is too much a aggregate value. If you keep the individual scores that says a lot more. At least it gives details for people to check which score they like. I want high read speeds since I use PowerRUN. People who do lots of backups might be interested in Write speeds. To be honest I don't know how many of the other values will be used in practice. But the File Read and File Write are accurate enough for comparisons.

Another option would be to use Card Speed (http://www.palmgear.com/software/showsoftware.cfm?prodID=48563). This will only give a read and write speed.

[CliePet]

> I really don't understand why n2fip keeps on complaining about VFS Mark...

Well, I speak only for myself...

The basic tests are not too bad, although not calibrated for the CLIE. The fundamental flaw is in how the summaries are reported.

For example, the read (and write) test is half done as small blocks, and half done as large blocks. The time for both are added together and reported as the "File Read" percentage based on timings made on a non-CLIE device a long time ago.

Reporting the individual numbers for each test is helpful, but I wish there were separate numbers for both cases.

----
The real problem is the total VFS Mark number (aggregate value).

It is a joke, technically flawed in several ways and generally misleading. If someone were to use it to tout their performance improvements, that would be sad... http://www.cliesource.com/forums/showthread.php?s=&threadid=15747

Why is it bad ?
It is an unweighted average of the speedup ratios.

"Unweighted", so it assumes you spend an equal amount of elapsed time performing DB Import as you do performing raw reads. Not realistic.

An average of the "speedup ratios" ???. Time does not work that way in this universe. It is somewhat bogus if the ratios are close to 100%, but it is extremenly bogus when number get over 200%
----
Fortunately the VFS Mark program is open-source, so you can see exactly what they are testing and reporting and fix it as appropriate. Thanks for the pointer for Card Speed. I am getting questionable write speeds on my tests. Not open source, so it is hard to know what they are doing.

FWIW: I wrote my own raw read/write program to compare with CLIEs and other PDAs (eg: PocketPC).

[CliePet]

An illustration on why the VFS Mark number (total/average) is bogus:

--------
How to boost performance of your day by 50% or more !

Let's say your day is split into:
8 hours of sleep
8 hours of work
7 hours and 57 minutes of other stuff, and
3 minutes brushing your teeth

Reduce your teeth brushing time to 1 minute, a 300% speed up !

Compared to speed of your new schedule to the old one:
sleep: 100%
work: 100%
other: 100%
teeth brushing: 300%

Average speedup = 150%

Congratulations, your now have 50% more hours in the day!

If you think this is logical, continue to use the VFSMark average.

[patrickl]

CliePet,

Yes as I said, I don't like the aggregat VFS Mark either, but the other testresults of VFS Mark (the program) really mirror quite accurately what other benchmarks you could do. I lost my notes (so I have no exact figures), but timing of starting apps with PowerRUN really gives about the same index as the File Read from VFS mark.

Where is your program? Maybe we could use that to test our cards?

[CliePet]

> ... gives about the same index as the File Read from VFS mark.

Agreed, of the VFSMark numbers that are often reported, the "File Read" and "File Write" are by far the most interesting.

However, as I mentioned, they mix small block reads and large block reads in one test result.

EruWare has added a cache to the latest driver, which is a good optimization.
BUT because of the way VFSMark works, this makes the File Read number less informative now than it was before.

For convenience, i/o tests often repeatedly read the same data over and over. Without a cache, it is a reasonable thing to do. When the driver is caching small blocks, it is artificial.

-----

My raw i/o test program is available here (updated version .02)

http://aibohack.com/clie/Time_RawVFS.prc

This is very similar to one half of the VFSMark file read (ie. large block reads)

This slightly newer version runs the test with 1K, 4K and 16K blocks. It still reads blocks over and over, but you can get a better idea where the cache is helping.

For this program: 1KB/sec = 1000 bytes/sec

-------

WARNING: gives very wacky results with the 1.1 driver especially with large CF cards (sometimes the cache slows things down). Gives reasonably consistent results with the 1.0 driver, but still slower than I would like.

BTW: The CF driver caching doesn't change writes -- and my 1GB SanDisk is still insanely slow to write to from the CLIE (ie. 7KB/sec for small blocks compared to 30KB/sec for a normal memory stick)

-----
I also wanted to report a real number (KB/sec) instead of an artificial percentage or aggregate total.
Numbers can be compared to other platforms (I have a similar Pocket PC version).

I know it isn't a fair comparison but the same 1GB SanDisk that runs pig-slow on the CLIE runs acceptably fast on a Pocket PC.
The Pocket PC is faster by a factor of 20X for small blocks, and 3X for large blocks.

I am still *hoping* future EruWare driver releases will close this gap, but it may not be possible because of overhead.

[n2ifp]

Well. at least Bill Gates didn't try to cripple the CF Slot!

For the most part, the CF works and speed doesn't seem to be an issue for me. Of course moving data between cards is slow!

[patrickl]

Thanks for the software. I'll try that.

[brahamt]

My Sandisk 128 CF card got 138. File writing, deleting and DB exports were below the benchmarks and everything else was above.

My MS got similar numbers and ended up at 133.

[patrickl]

I ran Time_RawVFS on my NX70 with a MsPro 512MB and got:
1kB:R=508, W=12
4kB:R=842, W=50
16kB:R=1013, W=200

(the dialog had "error" as it's title though, but that's probably since it's a copy from an error dialog)

[CliePet]

Originally posted by patrickl
> I ran Time_RawVFS on my NX70 with a MsPro 512MB and got:
> 1kB:R=508, W=12
> 4kB:R=842, W=50
> 16kB:R=1013, W=200

The slow write time (especially on small writes) is frustrating.

That's similar to the write speeds I am seeing for 128MB White Magic Gate sticks

The slow write time is unique to the CLIE, perhaps because they are updating the directory.

BTW: Smaller memory sticks appear to be a little faster on the writing.

BTW2: When I put the exact same stick into my Pocket PC (using a MS -> CF adapter) the write speed is 2 or 3 times faster.

Perhaps future write caching in the driver could speed up the CLIE version (but write caching is dangerous) ?

---
> (the dialog had "error" as it's title though, but that's probably since it's a copy from an error dialog)

Correct - laziness not a true error.

[patrickl]

Yeah it's kinda striking that you can get such a boost by using larger blocks. Do developers know this? For instance, imagine if PowerRUN uses a 1kB buffer. They could potentially increase read spee by 17 times just by using a larger buffer.


Actually I just tried:
- a 8MB blue stick:
1kB:R=383, W=30
4kB:R=716, W=123
16kB:R=881, W=330

-a128MB blue stick
1kB:R=372, W=15
4kB:R=704, W=64
16kB:R=912, W=264

So this is a lot faster at writing. These sticks were empty so that might matter a lot too.

But what makes me wonder most is the fact that read speeds don't seem to differ that much. Really PowerRUN became a lot quicker when I switched to the MSPro.

Besides, aren't the specs on these things that they can write at 1.5MB/s? So they are held back a lot by the Clie then.

[VLevyNYC]

Bottom line: is a 256MB CF or MSPro faster? Is there a comparable number in terms of MB/seconds for transferring files from PC to the card?

[DISenchanted]

Ok I know that it was decided that this was not a real example... But I am puzzled why my results are so much higher...

NZ90 with 3bit hacked nx73 drivers, PQI 40x 1gb card...

File create: 593%
File delete: 472%
File write: 284%
File Read: 309%
File seek: 536%
DB Export: 316%
DB Import: 646%
Record Access: 489%
Resource Access: 480%

VFSMark: 458 (tested 3 times 458/458/459)

512 Sony MS Pro

File create: 118%
File delete: 113%
File write: 20%
File Read: 201%
File seek: 256%
DB Export: 101%
DB Import: 356%
Record Access: 306%
Resource Access: 288%

VFSMark: 195 (tested 3 times 195/195/194)

[DISenchanted]

Figured I'd include the raw #'s as well...

1GB PQI CF

1KB: R=949, W=262
4KB: R=1254, W=862
16KB: R=1370, W=1861

512 SONY MSPRO

1KB: R=489, W=12
4KB: R=746, W=50
16KB: R=866, W=202

[CliePet]

> Ok I know that it was decided that this was not a real example... But I am puzzled why my results are so much higher...

> NZ90 with 3bit hacked nx73 drivers, PQI 40x 1gb card...
...
> File write: 284%
> File Read: 309%
> VFSMark: 458 (tested 3 times 458/458/459)
...
> 16KB: R=1370, W=1861

Cool, those are better numbers than I expected. I've heard good things about the PQI cards.

The "File write" number is much faster than other devices -- and it makes the VFSMark number sky-rocket in unrealistic proportions. That's close to 2X faster than my 30X Transcend card.

Write speed is PQI's claim-to-fame for cameras (where write speed is critical, read speed doesn't matter).

-----
> 512 SONY MSPRO
> 16KB: R=866, W=202

That's not out of the normal with other reported numbers. Some people have reported faster read times, but not that much faster.

In the (current model) CLIEs, the MSPro stick is still using a single pin for data (there is a faster 4 bit inteface which you can't use -- ie. you are paying for speed features you can't use).

CF cards use a parallel data bus, so with faster Flash ROM chips, a CF card should be faster than even the fastest MSPro stick in a CLIE.

Write speeds for MSPro haven't been that much faster than normal memory sticks, which surprises me. One of the selling points of MSPro sticks is a faster/guaranteed write speed - but maybe that only applies when used in 4 bit mode ??

-----
Bottom line:
Saster (more expensive) CF cards can win over MSPro sticks for speed. They are usually cheaper to.

Slower (and cheaper) CF cards will win hands down for price, but may not for speed.

Results vary greatly by brand.

[style]

NX80V

Kingston 512MB CF
VFSmark: 488

Sony 128MB MS
VFSmark: 158

[Eddy]

I am still confused as to get a really good fast running CF, & I don't want to shell out more money than that I have already. I don't have any digital cameras that use CF Cards, so I'm buying those CFs just to use with my CLIE, so far my SanDisk256 is doing alright, but the 1G SanDisk (well, I found out ways to work around it, to get it fully operational) but yet feeling the temptation to purchase some more new cards. With all those prices I've paid for the CF Cards, I can already have another Clie. :D
Oh, I'm going to try an 256MB SD Card (with the Panasonic adaptor to see if this would make any difference.) :)
Once again, all expenses are from my wallet. :(