Test the Stability of your CLIE

[BlueTooth]

Just came across this software to test the stability of Palm OS handheld. Download it here.
StabilityTest (http://www.handango.com/PlatformProductDetail.jsp?productType=2&jid=F4EAD31355BB9B2796FX35A3D2EFF257&platformId=1&siteId=1&productId=40511&sectionId=0&catalog=1&authorId=111593)

[Brokken]

Seems like a lot to pay to test something which you can empirically tell by simply using your PDA.

[s_n_m]

Nice wording.

[pnorman]

I just downloaded and installed the free (sponsored) version. It ran okay, and did NOT cause a hard reset (a caveat at startup).

Of the several tests it ran, one caught my eye: "Defrag". I was never sure of how much of a problem that was for solid state memory. One friend told me the term has NO meaning outside of floppies & HDs. Intuition tells me that, considering all the apps & files that go in & out of a typical Palm OS device, it has to have a cumulative effect of some sort ... and a performance degrading one at that.

Can anyone provide any info on that? Is there any sort of (safe!) defrag utility available?

[Atomic Chicken]

pnorman,

Your intuition is wrong - defragmentation has NO effect on solid state memory access times, cumulative or otherwise. The only legitimate performance gain that defragmentation can offer has to do with mechanical memory (such as hard disks and floppies) - where a read head has a mechanically related seek time aspect attached to it.

With solid-state memory, it is just as fast to access bytes which are next to each other in memory as it is to access bytes that are thousands of bytes apart in memory address space.

This "defrag" function sounds like a marketing gimmick to me.

Best regards,
Bawko

[Atomic Chicken]

P.S. - I am a consulting Electrical Engineer and Software Engineer by trade, so I DO know what I am talking about regarding this topic. I'm not saying this to be egotistical or anything, just to supply background as to the credibility of my previous statement.

Best wishes!
Bawko

[pnorman]

Thanks, that's precisely the sort of reply I was looking for. My knowledge of what's "under the hood" in these devices is sketchy at best. I assume from your reply that an app/file is always laid down into memory (Main or External) in a continuous block (or in contiguous units).

What puzzled me at first was the specific reference to "Defrag". Naive of me perhaps, but I tend to regard all Palm OS developers as"honorable" people (?!?). Consequently, I assume that there's some sort of "legitimate" usage of the term "Defrag" in such devices.

[Atomic Chicken]

pnorman,

Regarding what you wrote:

"I assume from your reply that an app/file is always laid down into memory (Main or External) in a continuous block (or in contiguous units)"

Unfortunately, this is not what I was trying to convey with my posting. The fragmentation of data is typically just as bad with solid state memory as it is with disk memories, it is just that solid state memory does not have the seek time limitations that disks do. Let me try to explain.

Imagine it in this way. You have a nice even row of "containers" (we will use an ice cube tray as the example) that you can put data (programs or files) into. In order to place data into these memory locations (the ice cube holes), you need to have a way to find which locations belong to which piece of data. This is the task of the "file system", which is part of the operating system.

Let's say that you store a large program in memory that takes up 1/3 of the ice cube tray. Then, you store a small program that only fills up 1 cube, then you store another large program that takes up a few cubes. Later, you delete the small program that only takes up 1 cube. You now have an empty cube between 2 rows of full cubes.

At this point, there are 2 ways of using the empty memory. The file system can either (a) wait for a program that only takes up 1 "cube" of memory before filling the single empty hole, or (b) place part of the program in the single empty cube, and the rest of the program in the cubes after the third program you put in the tray. The file system would then keep a "file allocation table (FAT)" which tells it which cubes belong to the data of any particular program, so it can find all the "cubes" of memory and link them back together again when retrieving the file. This is the way that all modern file systems work - with the files having the ability to be "scattered" throughout memory, or "fragmented" if you will.

The job of a "defrag" program is to re-organize the memory, moving the "cubes" around like a big puzzle, until all sections of each program are "contiguous" or "next to each other" again. This makes a HUGE difference on hard disks or floppy disks, because the mechanical read head does not have to move all over the surface of the disk, wasting time with each mechanical movement, in order to locate all of the scattered pieces of data that belong to a particular file. This can make a HUGE difference in performance for mechanical disks.

Solid state memory, on the otherhand, is different. It is just as fast to locate and retrieve data pieces that are right next to each other in memory as it is to locate dozens of data pieces scattered throughout the memory address space. This is why a "defrag" utility does not make sense for solid state memory, and why I am calling it a marketing gimmick.

Hope this all makes sense, I am trying to use as simple an analogy as possible :)

Best wishes,
Bawko

[pnorman]

I've had it "explained" by others, but NEVER so well ... and more importantly; in a manner that will remain in long term memory (contiguous or fragmented). Thanks for the carefully thought out explanation. I could probably now explain it to someone else, and with little if any garbling. (A bit for bit digital copy?). That's the acid test of a good explanation.

There are probably more than a few here at your level of professional competence. But "explaining" to a layman, and so well and with NO sneering condescension ... not much that of going around these days. THANKS.

[Atomic Chicken]

:)
You're welcome!

Remember - support your local chicken!
Bawko

[Hannibal]

Originally posted by Atomic Chicken
Hope this all makes sense, I am trying to use as simple an analogy as possible :)


Excellent clarification and analogy! In fact, I have pasted this into Memo Pad ;)

[XGlow]

...so then am I to understand that Defragging a Memory Stick is a waste of time??

[the_iceman]

I think Atomic Chicken makes a very good point. However I do believe there should be a small amount of clarification made.

I, when I was running windows 98 (pc talking about here) I had major issues, as most did/do, with keeping up a win98 system for good lengths of time (over a week) without rebooting. Part of the problem with many OS systems is the poor memory management. This should be one of the main focus areas of an OS ... to keep itself clean, however it didn't seem to be the focus of MicroSoft for some time and I do think the PalmOS does a decent job... but not the BEST job with memory management.

A program, while on a HD does in fact either utilize FAT (File Allocation Table) or can use NTFS. While this may depend on your type of windows operating system ... there are some subtle differences.

Not to get into a discussion with Filesystems, as this thread is geared for memory.... I'll try and keep my comments to the topic at hand.

I think your point ... the memory fragmentation in a PDA isn't important..... I'd have to disagree with you. I do feel the fragmentation can be important depending on what is running.
A .prc which is contiguous and is 50k will result, to the end user, the same speed in running as a 50k .prc file fragmented over 5 different areas.

I would say the same would not necessarily be true with a 2.0MB netfront 3.0 spread around 100 different locations in internal RAM (filling in the holes). The PDA still, no different than if the data resided on a Physical disk, must keep a 'tree' of pointers where the program's locations are in memory. When it needs to fetch the next piece of information and where it is located. A disservice is actually performed when a program is fragmented in memory. The more fragmentation in memory ... the more pointers/locations it must track which wastes even more memory (very little I must admit(< 1k).

An assumption seems to also be made that code being run on a PDA is 'good efficient code'. I've seen enough programs on windows platforms which cause severe memory leaks. The same can be said for the PalmOS world. There are enough programs which 'bleed' precious available memory. Fortunately for most of us... this bleed is basically limited during the execution of the program itself (providing it isn't allocating space for a a68k database).

Defragmentation Isn't as necessary for a Palm Device as it is on a PC. That much I can certainly agree with. I, myself, wouldn't mind to 'compact/reorg' main memory on my PDA probably once every 6-12 months.

With new handheld programs utilizing MS..... much of the data which was required to remain in RAM can now be placed on a MS and run from there (temporarily copying the files into Core RAM and then removed). This is a huge benefit to not have to defrag. You are using your MS like a hard drive... moving a program/files into ram... then when you are done with the program ... they are removed from ram.

This, however also poses a problem in itself..... that the memory management of the PalmOS handheld does a good job at keeping the memory 'clean' all the time. The PalmOS world has been created that all information resides in RAM. There has now been a huge switch. Now, with programs like PowerRun, ZLauncher and other utilities/programs..... We can now move our programs/databases out of RAM and onto a MS. The Memory management side of the PalmOS becomes much more critical now. with data flying in and out of RAM .... it is so very important to have a GOOD memory management system to not leave bytes/k 'hanging' when a program exits RAM. If not .... it is like lost-clusters on a Hard Drive. The space is allocated, but nobody knows what it is for. It becomes wasted memory which will basically not be recovered until a hard-reset is performed. Many posts have come about with 'corrupt memory' hard resets... etc. Sure a Hard Reset is a great way to remove any fragmentation in RAM. I don't think that is necessarily the solution. The solution is really to not necessarily have a defragmenter for the PalmOS... but a Memory Optimizer for the PalmOS... which is dedicated to keeping it 'clean and optimized by possibly defragging'.

I'd estimate, from a performance standpoint what a user might 'see' in a defragged memory in a PDA ... would be 1% average at best. However.... less pointers in RAM knowing where to go for the next series of instructions for the program while running..... well that is a different issue. That is more a waste of the OS and subsystem..... which is just CPU/IO(Yes even Memory addresses use I/O)/resource ticks against your handheld PDA. Less I/O it has to do to jump around and get everything assembled.... Longer Battery Life..... :). Would the battery life be substantial? Probably not. Maybe 5 minutes more per charge depending on the unit itself, and the amount of fragmentation.... but I would venture a guess there would be a very small benefit.

I've finally gotten to the point ... of moving almost everything off of RAM to my MS on my NZ handheld. I've found the following:

Slower Launch because it must temporarily move the files from the MS to RAM .... Great execution .... but less battery life.

Major downside to loading apps to the MS. It must read and write to the MS very often. The I/O to the memorystick is a huge battery-sucking process.

Anyway.... enough on this .... I do appreciate your very nice description of fragmentation and the ice-cubes! ... Very well put!

[the_iceman]

xglow,

I have heard of people having problems aftger defragging a MS.

Personally..... What I've done in the past ... is copy my MS contents to my HD on my PC... re-formatting the MS ... and then re-placing the files back on the MS. Files shouldn't be fragmented. Much safer to do this process than to defrag the MS. Just my $.02 about defragging MS.

[Atomic Chicken]

Greetings!

I have read the posting by "the_iceman" and "xglow" and would like to reply to a few things that have been said. First off, I would like to address the question by "xglow" about performance benefits of defragmenting a memory stick. It is true, as "the_iceman" states, that if a memory stick becomes extremely fragmented, the file allocation information tree may become complex enough to cause a SLIGHT degredation in performance. I think the figure of 1% quoted by "the_iceman" is probably fairly accurate. However, studies have shown that when it comes to computing technology, using typical applications, most users do not report a perceptible difference if the speed difference is less than 5%. In light of this, it is my personal opinion that defragmentation of a memory stick is a waste of time - and potentially dangerous to your data.

The potential danger to the data is due to the way that data defragmentation works. The data is moved from one location to another, and either before or after the move the file allocation table is updated. If there is a power loss, processor glitch, or someone removes the memory stick between these 2 processes, the result will be an incorrect file allocation table, and correspondingly corrupted data. This is why some people have reported problems after defragmentation.

I think that the method "the_iceman" recommends would be a safer alternative to traditional defragmentation, if you decide that you really do want to defragment your memory stick for whatever reason.

To "the_iceman" - thanks for the clarifications.

Best regards,
Bawko

[XGlow]

:)
Thanx Guys!
All of this information has a substantial value. Given the size of the MS, I will now do the copy to PC and then back again. That even SOUNDS safer. This all makes much more sense now that it has been explained in laymen terms! Much Appreciated!

Oh yea, I have found this thread to be MOST informative and very valuable information. Maybe one of the moderators could whip up some magic with the SUBJ and let it reflect something in the way of Defrag and Mem allocation? That might be a great help for people that might do a Search. I know "I" have become frustrated sometimes when the subject doesn't match the content... ...jus' a thought.

Thanx Again Guys!!

[the_iceman]

^5s Atomic Chicken... :)

[Atomic Chicken]

Iceman,

High 5's to you too, brotha :D

Or, in my case, it would be "High 4's", as we chickens only have 4 talons.....

Best wishes,
Bawko

[euroclie]

Originally posted by pnorman
Can anyone provide any info on that? Is there any sort of (safe!) defrag utility available?

Well, it used to be an issue with older PalmOS versions (up to 2.0), but since 3.0 the problem has been fixed and there's no such thing anymore as fragmentation in the PalmOS devices RAM:

Here's a quote from PalmSource Memory Manager documentation (http://www.palmos.com/dev/support/docs/palmos/Memory.html#996283):

In Palm OS version 1.0, individual storage heaps were limited to a maximum size of 64 KB each and the Memory Manager moved objects automatically among multiple storage heaps to prevent any of them from becoming too full. This strategy tended to decrease the availability of contiguous space for large objects. The version 2.0 Memory Manager abandoned this approach, increasing the availability of contiguous heap space; however, it still limited the maximum size of individual heaps to 64 KB each. Palm OS version 3.0 removed the 64 KB maximum size restriction on individual heaps and creates just two heaps: one 96 KB dynamic heap and one storage heap that is the size of all remaining RAM on the card.

Starting with Palm OS 3.5, the dynamic heap is sized based on the amount of memory available to the system<snip>

[yoopermjm]

Thank you Atomic Chicken.&nbsp; That was a most informative post and I always love to learn new things.

&nbsp;

[pnorman]

What initially drew me into the discussion was this: Some time back, I had to "restore" my T615 by HotSynching, after a total battery discharge. Prior to that, I had less than 1MB free memory. But after "restoration", I had quite a bit more. Around that time, I had run into the term "memory leak", as applied to MS Windows. That's the sort of phrase that sticks in one's mind but doesn't really explain anything. But it made me go "aha!!", and ever since then, I've regarded large scale loading & unloading of apps & files with deep suspicion.

[euroclie]

Originally posted by pnorman
What initially drew me into the discussion was this: Some time back, I had to "restore" my T615 by HotSynching, after a total battery discharge. Prior to that, I had less than 1MB free memory. But after "restoration", I had quite a bit more. Around that time, I had run into the term "memory leak", as applied to MS Windows. That's the sort of phrase that sticks in one's mind but doesn't really explain anything. But it made me go "aha!!", and ever since then, I've regarded large scale loading & unloading of apps & files with deep suspicion.

While memory leaks on PalmOS could probably reduce the amount of free RAM, I think that in most cases only the dynamic heap will be concerned by that problem, which means that the "storage" memory should not decrease in this case (remember all those talks about not the whole 8 or 16Mb of RAM being available to the end users? That's because part of it is reserved by the OS for the data heap, which is used by programs when they need to allocate some memory for internal storage purposes. The good thing is that only that reserved part of the memory should be concerned by the memory leaks...)

What could happen is that the backup program may not have backed up all your files (some of them do not let them be copied easily), or did not restore them. For instance, CardBackup can't restore the Wifi drivers, or some system patches. Also the builtin backup function of HotSync has the same problem, and do not backup all the files, which means you could have more free RAM after a restore regardless of any memory leak...

[pnorman]

"What could happen is that the backup program may not have backed up all your files (some of them do not let them be copied easily)"

That was my suspicion, but not having inventoried all that stuff, I'll never know for sure. I'm a real pack-rat, and am constantly moving stuff in & out. Using Megalauncher, I put as many apps as possible into the Memory Stick, but that 16MB limit STILL cramps me.

I finally "solved" that problem with my T615, by making an incremental upgrade to a T665. I HotSynch with the T615 exclusively, and populate the T665 by IR and MS swapping. After ~4 months, I'm STILL working on a suitable division of labor between the two, but I'm getting there.

Next year around this time, Sony ought to have a suitable upgrade model for me (T form-factor or close to it, 64MB RAM, Palm OS 7(?), 400 MHZ, VG). Ability to access a CF card would be nice, if only with a MS>CF kludgey adaptor.