Purging hands

[dejankol]

Hi,

I'm using PT3 for a while without any problems. However, this morning (4 hours ago), I wanted to purge some old hands (12+ months old), so I picked a month from last year. The amount of data selected for purging contains probably around 50K-60K hands from cash games. The procedure is simple and I did everything I had to, but, since I started the procedure, PT3 is not responding. I know that it's a slow process, but is it really taking that much? Should I, maybe, terminate the program (am I risking to get a damaged DB in that case), or I should be patient? I have no idea what to do now. Can someone help me on this?

Just to mention, I'm running PT3 on a decent PC, and I did DB (800K-900K of hands) housekeeping on regular basis.

Thanks.

[WhiteRider]

Purging is a pretty long, involved process - I'd suggest letting it run, terminating the process could cause problems. The estimated time is much longer than it will actually take.

[dejankol]

Thanks for your answer. The problem is that I didn't get any estimation. Now it's been 6 hours since I started the purging process and the PT3 is not responding at all. I will wait, but I have a feeling that this is going to last forever.

And my question still stands. How much time it takes for the prog to purge 50K of hands? I don't want to deal with old information, so I want to delete some old hands from time to time, but if I need 5 working days per month for keeping my DB in good shape, then it is very disappointing.

Today I wanted:
1. To purge some old cash sessions;
2. to purge some old tournaments;
3. to do DB housekeeping;
4. to play some of my favorite Sunday tournaments.

Well, the tournaments have already started, and I'm still at the first step of my today's plan.

[WhiteRider]

How long it will take really depends on a lot of factors - removing that many hands is a big job, though, and will take quite a long time. The development team are aware that purging takes time and I hope that they will be able to find a way to optimize it at some point, but it's a complex process as every interlinked hand and player for each session has be identified and removed.

[dejankol]

Thanks for your kind response. Well, after I woke up this morning I found out that purging is still in progress. I even got the estimation of when the process will be completed (in 80 hours). This is still a big problem. It's not only that I'm locked of playing my games for few days, my computer is being slowed down and it's making difficult to work on anything else during this time.

Now I can see that there's nothing I can do to delete the outdated records from my DB, and I don't like the idea of having information about other players based on their year+ old performance. Players do change their style, they are learning through the time and the way how someone played a year ago can be (and probably is) significantly different then today. In addition, if I don't purge my hands from time to time, I will end up with a very huge DB, which will, finally, lead to a very poor performance (I'm already experiencing slow-downs caused by the size of my DB). Generally, I like using PT3. I find it to be a very powerful peace of software, but I'll have to try HEM to see is it any better regarding mentioned issues.

Once again, sorry for bothering you. I hope you will find the way to improve your software.

[WhiteRider]

Large purges do take time - it won't take 80 hours, the estimate is always too high.
You may find something like this useful: Tutorial: Managing Multiple Databases.
If you're purging a lot of old hands like that you'll probably find it faster to re-import the hands you want to keep to a new database, either from the original files or by doing a selective export by date.
That said, I do hope that purging speed will be improved too.

[dejankol]

OK, I have one more question and I won't bother you any more. If I, for instance, create 12 DBs, one for each month in last 12 months, and if I import adequate hands into each of those DBs, and if I set my HUD to drag info from all of these 12 DBs, will it slow down the HUD?

If I organize my data in this way, I could easily start a blank new DB for every month and stop using (and delete) the oldest one. Also, housekeeping would be necessary only for the latest DB that's still accepting new hands.

If 12 DBs would be too much for HUD to deal with, what about 6, or 4?

If this is possible without performance issues, then it would be a nice way to deal with the problem.

Thanks again.

[kraada]

12 will obviously (and necessarily) run slower than 4 just because of the extra work that PT3 needs to do. Whether or not it will be feasible will depend heavily on the size of each database, the speed of your hard drive, and the general speed of your computer. 4-6 databases shouldn't be a problem for most people, but I don't know anybody that's tested as many as 12 so I can't say for certain how drastic the slowdown might end up getting.

[dejankol]

I decided to use 4 DBs. In each DB I'm going to keep hands played during a three months period (250K - 300K hands each). One of the reasons is also that I don't want to use too many DBs not only because of the HUD performance, but because of the table scanner software that I'm using too. I don't want my table scanner connects to 12 DBs during scans because it will surely lead to additional problems. Once when I set up these DBs I assume there is not going to be too much work to keep them in good shape and I will no longer ever have to purge any hands.

I want to thank for the help to both of you, you've been helpful.

[kraada]

You're very welcome. Given your situation that sounds like a good choice in terms of setup.