[kwlug-disc] 16GB flash-memory pen transfer speeds up then corrupts files

[Rick]

I got this 16GB flash-memory pen-drive from eBay

   http://cgi.ebay.ca/ws/eBayISAPI.dll?ViewItem&rd=1&item=350126058238&ssPageName=STRK:MEWN:IT&ih=022

and I have been trying to use Linux to copy files. At first, all seemed
well.  But, then, I did some hard testing by copying over 5GB to 15GB
all at one go. When I do larger copying, I always get file corruption at
some point during the copy. I have tried the following partitioning:

   (1) with /dev/sdX1 as Linux ext2
   (2) with /dev/sdX1 as FAT32
   (3) with /dev/sdX as a single FAT32 "superfloppy"

By the way, the "superfloppy partitioning" is when the memory-card
or pen-drive doesn't have any partitions. The entire media is
a single partition. You format with:
   # mkfs.vfat -F 32 -I /dev/sdX
      ... note the "-I" argument
and you mount with:
   # mount -t vfat /dev/sdX /mnt/...
Notice that there is _no_ partition table number.

The superfloppy setting was how the pendrive had arrived and, then,
"fdisk -l /dev/sdX" spits out a weird partition table (which, based on
my readings, is acceptable). In fact, I have a new "MP4 player" device
which also reports a similarly weird partition table.

I'm not sure if this file corruption is from having older drivers or
if this pen-drive is broken.

I read that I should set /sys/block/sd.../device/max_sectors down from
240 to, say, 32. That setting helped and, once, I managed to get 12GB
copied. (Or, at least, "du /mnt/..."  says that 12GB got copied and the
"ls" didn't show obviously corrupt files.)

I reformatted this 16GB pen-drive from a WinXP machine. Then, while
still on WinXP, I tried copying a 5GB directory. But, again, I got
corruption ... this time, after only around 2GB of copying.

Now here is the weird part. Both on Linux and on WinXP, copying proceeds
at a slow rate but, then, suddenly speeds up. That is, WinXP said that
it would take 40 minutes to copy everything and, halfway through, the
remaining time dropped from under 30 minutes to just 4 minutes
... which, I believe, is when corruption started happening.

As for Linux, I believe that the transfer rate suddenly sped up because
I was monitoring disk i/o with the "gkrellm" system-monitoring tool.
Paralleling what happen in WinXP, the "gkrellm" graph shows a sudden,
sharp jump in the transfer rate partway through the copying.

I'm not sure if the WinXP has the latest drivers for pen-drives
(or do drivers remain unchanged at the 16GB level for flash-memory
pen-drives?) Also, perhaps my version of Linux might not have the newest
drivers.

I'm guessing that I most like have a broken pen-drive ... but the eBay
seller has hundreds of buyers and no negative reviews (at least
in the time frame that eBay is reporting which, I've read, is 30 days,
down from 90 day).

Also, I did this formatting operation at one point:
   # mkfs.vfat -c -v -n 16gPen -F 32 -I /dev/sdj
Note the "-c" flag ... which will "Check the device for bad blocks
before creating the file system". (I assume that "badblocks"
is being run.) And "mkfs.vfat" did not report any errors.

Any opinions on the state of the pen-drive ... or, better yet,
comments on what I'm doing wrong? Maybe my version of Linux is lacking
something to properly handle 16GB of flash memory? If so, maybe
my initial efforts on Linux have changed formatting in the flash-memory
and WinXP isn't smart enough to fully reformat what Linux did?