mds_storage over capacity

[Fyrd Argentus]

I seem to have exceeded the allowed storage limit on an MDS_storage variable. Now I can't even load it to break up the data into pieces.

I have PM'ed Mur for help.

Any solutions on how to rescue this data short of intervention by Mur?

p.s., Printed out the data is/was/should be about 10 pages of text. Very surprised this hit the limit....

Edited September 4, 2011 by Fyrd Argentus

[Chewett]

Wha??? so this isnt holding user data?

Please dont tell me this is holding quest text, that isnt going to change each time.

[duxie]

[size=3][font=verdana,geneva,sans-serif]i'm not sure how your 3d quest is storing data, and what type of data, so i wont be giving any predictions, but from what i saw - it is dependent on user's "storyline", or even on community storyline (dst came with this issue at her first try after we had it for few times). at first i would check what type of info i am storing, and the type of the storage. oh well... just one guess... you are not storing the whole "path" of the text into some sort of array for each person? [/font][/size]

[Fyrd Argentus]

Well, it's complicated.

A DIFFERENT storage data holds the player's x,y,z coordinates.

The file in question holds a list of populated coordinates and the text seen at each coordinate. It also holds data of who edited the scene last, but that is expendable. That's what I want to strip off so it will be down to acceptable size.

This can and will change as I update and modify things, but not just due to regular quester moving through. I have had "helper" editors in the past, and expect to have in future.

So what I'm looking at is about 200 data triplets, which should have been just data pairs. Each element is a string text.

p.s. - I was adding scenes during the time that it got harder and harder to load, now nobody can load it. Used to be intermittent, now is constant. Or such is my impression.

Edited September 4, 2011 by Fyrd Argentus

[Chewett]

Yeah, see really it shouldnt be holding that much data, especially not fixed strings as thats what the content sections were for.

[Fyrd Argentus]

That would be too static for the purpose.... Oh well....

[Maebius]

I have not tried this quest yet, but off the top of my head, depending on what changes, would you be able to use a template in Contents, and String-replace whatever segments may change? (I';m still trying to work out how Templates work, but appear to have the ability to hold values like {{REPLACE_THINS}} tags for semi-dynamic output?

Alternately, break your 3D array of coordinates into 2D arrays? So instead of one large Storage value, have a different Storage for each "row" (or column, or Z, or whatever). Might be a bit too much re-defining of what holds the @Storage, but might allow larger memory storage, if you do something like this horrid pseudo-script:

if (Player_X_coord == 1) { @va = @storage[X1][Y][Z]; }
if (Player_X_coord == 2) { @va = @storage[X2][Y][Z]; }
if (Player_X_coord == 3) { @va = @storage[X3][Y][Z]; }
echo "You are at location ". @plcoordinates. " and see: ". @va " here.";

[duxie]

[size=3][font=verdana,geneva,sans-serif]if i understand the idea of Fyrd, the possible solution would be to encode/zip text strings before storing them and unzip on displaying. there you should go on research about zipping text strings. might decrease memory usage for few times.[/font][/size]

[size=3][font=verdana,geneva,sans-serif]but that's not only your problem anymore. every quest will start getting the same error as the storage is common. this should be normal, as there are more quests created. old/invalid quests should be removed and the storage increased... also, would be great if it would be documented somewhere (maybe it is, but i cant find) how the storage works, i.e. what is the max size and per whom. player/quest/common to all/etc. so it would be more clear for quest creators what type/quantity of information they can operate with.[/font][/size]

[Rendril]

Is the memory being exceeded just by loading the variable? If not, write a small segmented output to retrieve the data.

[Fyrd Argentus]

Thanks for all the advise, but I'm in over my head.

Symptoms - for a time we were all getting intermittent "white screen hang" when accessing the clickie. Now it's always. On another clickie, simply calling the MDS_storage variable gives a memory allocation error.

So yes Rendril, I'm blocked at the point of just retreiving the data.

I don't know how to zip variables, and don't know if that would actually decrease the allocation the system gives to my work.

Yes, there are simple work-arounds now that I know what the memory limit is. I can dump 33% of my items right off the bat, and go to modular memory blocks if I have to -- break it up and keep it simple.

I have about 120 of the data pairs in archive dump form - could reconstruct with some work. Trying to save another 80 of them... But no way to get at them now.

[Fyrd Argentus]

Rendril, Mur basically referred me back to you. Please let me know if and when you give up, so I know when I need to pursue some other avenue of reconstruction.

Thanks.

[Fyrd Argentus]

Okay, I've begun some reconstruction and discovered the root of the problem. Every time my code checked @mds_storage to see if there was an entry for a coordinate, it saved that space. So, I was unintentionally building up a list of blank spaces that players had visited. This was getting indefinitly longer than my list of true entries.

So, I need to keep a list of REAL coordinates and do something like

IF (coordinates are on the real list) { see what the real entry is in storage}

[Fyrd Argentus]

Okay, maybe I didn't need to do this - maybe I did. Can anybody say whether accessing an index of an array in mds_storage -- WHICH HAD NOT PREVIOUSLY BEEN DEFINED -- reserves that spot in memory?

I've fixed my code so it doesn't matter, along the lines I suggested above, but I'm sure we'd all like to know..... Anybody?

[duxie]

[size=3][font=verdana,geneva,sans-serif]not sure if i understood correctly, but...[/font][/size]

[size=3][font=verdana,geneva,sans-serif]could you show an example of your IF statement? i mean, how do you check if the index exists in array? no offence, just interested: you use "isset", or "array_key_exists", or what?[/font][/size]
[size=3][font=verdana,geneva,sans-serif]you have 3D coordinate system, how do you define the key of such array? is the array multidimensional or you just put the key as a string, i.e. "xyz" => "some text"?[/font][/size]

[Fyrd Argentus]

I define a coordinate as a text string @vx.'-'.@vy.'-'.@vz then use this as the index for an array which has the contents of the site.

So if I look up @vc=array[above-text-string] and it is a not yet defined, i.e. @vc=="", does this create the storage space for it?

I think I was creating datafiles every time somebody wandered into new empty spaces.

A path out to 10-10-10 might take only 30 sites, but it "encompases" 1000 empty spaces....

btw - I actually stored the text string as data as well as using it as index value - so multiple fields in storage array. I got rid of a third field I didn't need any more.

Now, I have another, different array that is just the list of coordinates and I can check to see if a new spot is "on the list" before trying to call up the stored site info.

Edited September 13, 2011 by Fyrd Argentus

[Rendril]

Accessing the index of an array does not reserve memory for it, however, it is good practice to perform a check that it exists anyway.
Something like this:
[code]
if(isset(@storage['some_key'] )) {
@va = @storage['some_key'];
}
[/code]

Note: you can do the check without isset() but it will throw a notice level warning, in the case of MD script it doesn't matter though because it gets squelched.