Message Boards Message Boards

0
|
5736 Views
|
11 Replies
|
3 Total Likes
View groups...
Share
Share this post:

Create named empty lists, name coming from another list?

Posted 6 years ago

Is it possible to create named empty lists, the name coming from another list of names? As an example lets say I have a list consisting of words; names={one, bus, tree, townhouse...} and I want to now create empty lists called onetmp={}, bustemp={}, treetemp={}, townhousetemp={}. I want to append temp onto them all to distinguish them from their original names as they also appear elsewhere in my program. I want these named lists to be kept in a separate list so they can be called in turn by index and be able to add information into them. The example below doesn't include the equal sign when executed so In effect there is no named list created. Any help would be appreciated.

names = {"one", "bus", "tree", "townhouse"};
Table[names[[q]] <> ToString[temp = {}], {q, 1, Length[names]}]
POSTED BY: Paul Cleary
11 Replies
Posted 6 years ago

First of all I'd like to thank everyone for all the time and effort you have given this problem it is much appreciated, now I am not going to lie I am still trying to implement the ideas that have been suggested to see if they can be useful in the ultimate endgame of my problem. and so perhaps if I describe in a bit more detail what I am trying to do and give a real life example of the data I am working with someone may be put me on the right track. First the text file is imported as

raiddata=Import["raid1.txt", {"Text", "Lines"}];

Length[raiddata] = 151166 p.s. this is a small file. and raiddata[[19765]] is

"[Sun Jul 22 18:18:58 2018] Rokani pierces an Arcron dread magus for 11787 points of damage."

The data contains references to an unknown number of mobs, (a computer controlled entity) neither do we know what they are called, it also contains reference to players by their avatar names also an unknown amount and unknown names. As part of the parses function we have to identify this information, though the number of players isn't too important. and associate all those players that have interacted with each mob but only by the damage they have done to it and ultimately the type of damage. Different classes have different types of spells, i.e. direct damage and DOT's (damage over time), then there are the melee players, they just stab poke and bash kind of thing. The following is a small random sample of text from the file but chosen to give some idea of what is what.

"[Sun Jul 22 18:06:55 2018] Hsishi's eyes glow with the strength of the forest.",
"[Sun Jul 22 18:06:55 2018] Khen snarls, filled with the power of the predator.",
"[Sun Jul 22 18:06:55 2018] Blackjaw`s warder's eyes glow with the strength of the forest.",
"[Sun Jul 22 18:15:17 2018] Seibas begins to cast a spell. <Crown of Roses Nimbus>",
"[Sun Jul 22 18:16:20 2018] Daktarr has been delegated Raid Main Assist.",
"[Sun Jul 22 18:16:43 2018] LOADING, PLEASE WAIT...",
"[Sun Jul 22 18:16:56 2018] Fidden slashes a sepulcher skeleton for 20105 points of damage.",
"[Sun Jul 22 18:16:56 2018] A sepulcher skeleton has been slain by Fidden!",
"[Sun Jul 22 18:16:58 2018] You have entered an area where levitation effects do not function.",
"[Sun Jul 22 18:16:59 2018] Shadisar pierces a bile golem for 1311 points of damage.",
"[Sun Jul 22 18:16:59 2018] A bile golem hits Tankpaw for 4673 points of damage.",
"[Sun Jul 22 18:16:59 2018] Khoros slashes a bile golem for 3939 points of damage.",
"[Sun Jul 22 18:16:59 2018] A bile golem has taken 111723 damage from Mesil by Decay Effect III.",
"[Sun Jul 22 18:16:59 2018] A sepulcher spectre has taken 111723 damage from Mesil by Decay Effect III.",
"[Sun Jul 22 18:16:59 2018] An elementalbone skeleton bashes Dablubb for 2254 points of damage.",
"[Sun Jul 22 18:16:59 2018] Shadisar hit a bile golem for 7695 points of non-melee damage.",  
"[Sun Jul 22 18:17:03 2018] An Arcron dread magus begins to cast a spell. <Glowing Incandescence>",
"[Sun Jul 22 18:17:04 2018] Sanae bashes a sepulcher spectre for 1920 points of damage.",
"[Sun Jul 22 18:17:06 2018] Blackjaw`s warder slashes a sepulcher spectre for 4261 points of damage.",
"[Sun Jul 22 18:17:11 2018] Felrynn performs an exceptional heal! (171082)",
"[Sun Jul 22 18:17:11 2018] Felrynn hit an elementalbone skeleton for 109141 points of non-melee damage.", 
"[Sun Jul 22 18:17:11 2018] Felrynn delivers a critical blast! (109141)",
"[Sun Jul 22 18:26:09 2018] Drusella Sathir smashes Mizar for 23023 points of damage."
"[Sun Jul 22 18:26:09 2018] Drusella Sathir bashes Mizar for 20343 points of damage."
"[Sun Jul 22 18:26:13 2018] Sarryn slashes Drusella Sathir for 5390 points of damage."
"[Sun Jul 22 18:26:18 2018] Drusella Sathir has taken 30358 damage from Mesil by Cytotoxic Wounds Rk. II."
"[Sun Jul 22 18:26:18 2018] Drusella Sathir has taken 90249 damage from Stevnec by Pyre of the Shadewarden Rk. III."
"[Sun Jul 22 18:26:19 2018] Wevz crushes Drusella Sathir for 10568 points of damage."

The first two lines although they don't contain parsable data they do contain two players names, names are always found immediately after the time and date stamp. The 3rd line is reference to another player but more specifically his warder (this is a pet and will do your bidding). The warder can be parsed but in the end any damage it does is added to its owner's damage. The 4th line is fairly irrelevant but it does contain reference to a spell name, line 5 and 6 are totally irrelevant. Fidden is relevant as he has done damage to a mob, the mob being "a sepulcher skeleton" and he has slain a mob. (A search for "has been slain by" identifies all mobs except maybe the one who is too hard to fight and the raid wipes, so you can't identify all mobs by the slain tack tick. Also if you look at the two lines with Fidden "a sepulcher skeleton" and "A sepulcher skeleton" are the same mob, one is capitalized as it is the first word on the line the other is not the first word, these have to be associated together, the same with the "a bile golem" and "A bile golem". A relevant line is where Khoros slashes but "A bile golem hits" is not because it is the mob doing the damage and isn't parsed. The line with "A sepulcher spectre has taken 111723 damage from Mesil by Decay Effect III.", is very relevant it contains the mob name the damage done the players name and the name of the spell used, that particular line identifies a DOT playing character and as such needs special attention because over the fight of say a Raid mob, can have up to 20 different DOT's in play at one time each doing its own damage, this needs to be kept separate and summed individually and then finally totaled together along with any other damage done that are not DOT's and their pets if they have used one. "Drusella Sathir" is mentioned because she is the Raid mob, the lesser mobs can be dealt with by a few players the named mob is a different ball game, the encounter can take upwards of 45 minutes and all 54 people in the raid, and all 54 can "wipe" i.e die within a few seconds up to now I have parsed the named with over 2.5 billion hit points. and finally if that wasn't enough the last and most important part of the parser is to calculate the DPS or damage per second, so we have to identify the time stamp when a mob is encountered first and then when it dies from that we get a time interval for that mob say 32 seconds for the lesser mobs and each person associated with that mob has done so much damage during that time so a dps can be calculated per individual, it becomes a bit harder when an individual might not have been on that encounter for the entire time, they may be part of 2 or 3 separate ones. So as I have previously mentioned I have made an attempt to do all this with a fair modicum of success, I am still having difficulty in identifying the lines where the mob hits players, there doesn't seem to be a consistent thing to look for. My parser as it stands takes a good 10 minutes to complete this file due to the number of complete passes through the file, hence my original question about creating named lists that could be addressed by index, so on as few passes as there are mobs I can put into each list only that data relevant to the mob in question which incidentally I can do now just by creating a table of empty lists.

I do have a question, is there a way to access each line of data in the main file and compare it to say a name without using a Do loop? For example I identify the names of the mobs by this code

mobqnty = 
 Reap[Do[If[StringCount[raiddata[[q]], "has been slain by"] == 1, 
    Sow[raiddata[[q]]]], {q, 1, Length[raiddata]}]]; mobqnty = 
 Flatten[mobqnty[[2]], 1]; pos = 
 Reap[Do[p = Flatten[StringPosition[mobqnty[[q]], "has"]]; 
   pm = StringTake[mobqnty[[q]], {28, p[[1]] - 2}]; 
   ct = Flatten[ToCharacterCode[Characters[pm]]]; 
   If[Count[ct, 32] >= 1 && StringCount[pm, "pet"] != 1, Sow[pm]], {q,
     1, Length[mobqnty]}]]; pos = Flatten[pos[[2]]]; mobqntty = 
 Tally[pos];

which gives me this list.

{{"A sepulcher skeleton", 3}, {"A bile golem", 
  2}, {"A sepulcher spectre", 2}, {"An elementalbone skeleton", 
  2}, {"An Arcron lifter", 3}, {"An Arcron dread magus", 
  2}, {"A pyre golem", 1}, {"A Krellnakor scavenger", 
  3}, {"A Wulthan crusader", 2}, {"A hexbone skeleton", 
  1}, {"A Krellnakor enforcer", 1}, {"An Arcron delver", 
  1}, {"An Arcron helot", 1}, {"A Fereth appraiser", 
  3}, {"An Arcron firebrand", 1}, {"A Krellnakor filcher", 
  1}, {"A Wulthan thief", 1}, {"A bloodthirsty gnawer", 
  2}, {"A hexbone protector", 2}}

Where we now have the names of mobs and how many of them were in the encounter, after this I create a list of empty lists as such

nametemp = Table["{}", {i, 1, Length[mobqntty]}] (thanks to Marco who gave me this idea)

and finally

q = 1; While[q < Length[mobqntty] + 1, 
 nametemp[[q]] = 
  Reap[Do[If[StringCount[raiddata[[w]], mobqntty[[q,1]]] == 1, 
     Sow[raiddata[[w]]]], {w, 1, Length[raiddata]}]]; 
 nametemp[[q]] = Flatten[nametemp[[q]][[2]]]; q++]

That is partially the data as I also have to look for the non capitalized version of the name too. At the end of this I have a list of lists the complete list called nametemp whos sub lists can be referenced by nametemp[[i]] where everything now in nametemp[[1]] contains every line that contains "A sepulcher skeleton" or "a sepulcher skeleton" that particular list is 670 lines and Drusella is 43102 lines, to complete this task to this point including reading in the file takes 10 seconds a huge improvement on my old 10 minutes. I am certain it is not good programming but I'm not a programmer by profession.

I do appologise for the length of this post.

Regards

Paul.

POSTED BY: Paul Cleary

Paul,

The suggestions you have gotten so far have all been all excellent. I will add one more idea in the mix in case it might better suit your problem.

I like to use the assignments in MMA to make a dictionary. For example, you can enter

junk["one"] = {12}
junk["two"] = {4,8}

You can see all of your assignments with ?junk or Definition[junk] or programmatically with DownValues[junk].

The advantage is that you do not need to create new symbols manually or manage them. Using Marco's example (but making a string out of it since I think you said you are handling large strings)

Set up some lists:

In[6]:= names = RandomSample[WordList[], 10]

Out[6]= {"steps", "highlight", "puppeteer", "cleanness", \
"credential", "nonstarter", "ablaze", "telegraphically", "stoppage", \
"heads"}

In[7]:= tableentries = 
 Table[StringRiffle[RandomChoice[names, 4], " "], 20]

Out[7]= {"cleanness cleanness ablaze heads", "telegraphically \
nonstarter credential credential", "cleanness credential \
telegraphically stoppage", "stoppage heads heads telegraphically", \
"puppeteer ablaze nonstarter telegraphically", "telegraphically \
stoppage credential nonstarter", "telegraphically steps ablaze \
cleanness", "credential stoppage ablaze credential", "heads \
nonstarter puppeteer stoppage", "cleanness ablaze highlight \
cleanness", "ablaze heads nonstarter highlight", "ablaze nonstarter \
heads heads", "stoppage nonstarter steps heads", "cleanness cleanness \
ablaze telegraphically", "nonstarter telegraphically credential \
stoppage", "ablaze puppeteer nonstarter cleanness", "highlight ablaze \
cleanness cleanness", "puppeteer cleanness puppeteer ablaze", \
"stoppage nonstarter stoppage highlight", "credential heads \
credential ablaze"}

To create the dictionary programmatically create a function to assign the dictionary value, testing if its the first time it is used -- either set the first value or just append the latest value.

tmpify[x_String, val_] := 
 If[Head[tmp[x]] === tmp, tmp[x] = {val}, 
  tmp[x]  = Append[tmp[x], val]]

Now you can use it by first clearing all old entries in tmp and then mapping the function over the table of values and Map it a second time over the list of names to give the location of every name in every string in the table:

Clear[tmp]

Map[Function[name, 
   MapIndexed[
    If[StringCount[#1, ___ ~~ name ~~ ___] > 0, tmpify[name, #2[[1]]],
       Nothing] &, tableentries]], names];

This is the result:

enter image description here

This approach would be very fast when doing many lists and it would be easy to reference the result either with

tmp["steps"]

or

tmp[names[[1]]]

I hope this helps.

Regards,

Neil

POSTED BY: Neil Singer

Summing up the linked topic:

ToExpression[
  # <> "temp"
, StandardForm
, Function[symbol, symbol = {}, HoldAll]
]&  /@ {"one", "bus", "tree", "townhouse"}
POSTED BY: Kuba Podkalicki
Posted 6 years ago

Dear Marco

Before I turned in last night I had almost solved the problem, I had arrived at this

nametemp = Table["{}", {i, 1, Length[namestally]}]

which as you already know creates a named list of empty lists. This was perfect as I could then access each list by it's index, it wasn't necessary for it to have any name as the index number was associated to the names elsewhere. This also meant I didn't have to append any temp to anything (this was something I thought I would have to do so there would be no conflict between variable names and the same names elsewhere).

My goal in the end is to write what is called a game parser, when I say write I have already done it but it is very inefficient and slow mainly because I have to scan the full file each time for every mob as they are called, and then do full scans each time for every person (up to 60). I still have to do as many full passes as there are mobs, but each one now has it's data in one of these lists and this sorted data may only contain information relating to 1 to 3 or 4 different people. So to fully parse that mob to see who has done damage to it and how much only requires scanning a file of 100 lines or so up to 4 times which is a huge saving on my previous attempt.

I am reluctant to post any actual data on these forums but if you wished to see exactly what it looks like don't mind emailing you directly a text tile I have a small one with just over 151k lines, a few mb's.

Regards

Paul.

POSTED BY: Paul Cleary

In order to handle cases where those symbols already have values you need to incorporate the third argument of ToExpression, see:

POSTED BY: Kuba Podkalicki
Posted 6 years ago

Thank you for the link Kuba, I look forward to reading it.

POSTED BY: Paul Cleary

Dear Paul,

I am not sure whether the question is about realising it with procedural programming or whether at the end you just want the result. Also, I don't have the data you use. I will make it up according to what I understand. Suppose we have some random words from the dictionary

ClearAll["Global`*"]
names = RandomSample[WordList[], 10]

enter image description here

Now I make a "large" table with 20 lines. I could do millions of course, but it is difficult to post here:

tableentries = Table[RandomChoice[names, 4], 20];
TableForm[tableentries]

enter image description here

We now make a list of as many empty lists as there are "words" and then a list of these words with "temp" appended:

dummy = Table[{}, {i, 1, Length[names]}];
dummynames = ToExpression[# <> "temp"] & /@ names;

Now we can see where the words come up in the large table:

Do[AppendTo[dummy[[#]], k] & /@ Flatten[Position[names, #] & /@ tableentries[[k]]], {k, 1, Length[tableentries]}]

This is still procedural programming, but it is easily changed to functional style. Now we get everything together:

Set @@@ Transpose[{dummynames, dummy}];

Now you can ask for the lists:

degeneratetemp

enter image description here

propertiedtemp

enter image description here

Note that numbers are repeated if the word comes up several times in a row. If you do not need that you can use DeleteDuplicates to clean that up.

I have put in no thought of making this efficient at all. Also, I am only on a mobile device right now so cannot test this as much as I should. In fact, on a single CPU this will take about 15 minutes for 1000000 lines and 6 columns on a single CPU. ParallelDo could make this faster.

On a million lines this runs in about 26 seconds on my machine:

Do[Flatten[Position[MemberQ[#, names[[k]]] & /@ tableentries, True]], {k, 1, Length[names]}]

So, counting millions of lines of code should be quite possible.

Cheers,

Marco

POSTED BY: Marco Thiel
Posted 6 years ago

Dear Marco

I appreciate your time on this and indeed we are getting closer to my goal, those list's are created as you say and I can append to them by specifically writing into them, but my goal is to add into each list many thousands of lines of text that is parsed from a much bigger list where some lines of text contains the name of the created list. (if you see what I mean). Secondly there could be up to 60 of these created named lists, non of which I need to know the names of nor do I want to see them and I want it to be all automatic. So in a nutshell I have a large file consisting of multiple millions of lines of text, within these lines of text are anywhere from 1 to 60 names, the names can be extracted as they always appear at a certain position in the line and this forms the basis of my named list. I now want to create as many empty lists as there are names and on the second pass through the large file extract only those lines that contain the name and put it in the newly created temp list so at the end of the second pass each of the lists say onetemp now contains only those line that also contain one (one is a bad example I know) the names would be actual names of characters. So I need to be able to index the list of lists and then add information into them. Sorry if this is getting long.

Edit

If we add this to the line

nametemp = Map[Set[#, {}] &, ToExpression[# <> "temp"] & /@ names]

I can index nametemp as in nametemp[[1]] and use Reap and Sow and it does seem to work, it seems to add it to what starts as an empty list, yet if you directly type the name of the list it is still empty. Wonder if this is intended?

Cheers,

POSTED BY: Paul Cleary

Dear Paul,

all the lists are in fact created. If you just type

onetemp

you will see that it gives an empty list. If you want to add something to the list you can do:

AppendTo[onetemp,5]

and it will add 5 to that list. If you then execute

onetemp

it will be a list with 5 as the only element. I guess that the output was a bit confusing. You can suppress it with a ; at the and of the line.

Alternatively, this works, but is longer than my original solution:

tempnames = ToExpression[# <> "temp"] & /@ names; 
Do[tempnames[[i]] = {}, {i, 1, Length[names]}]

Also, procedural programming is discouraged.

Cheers,

Marco

POSTED BY: Marco Thiel
Posted 6 years ago

Hi Marco

Thanks for the quick reply, however, it doesn't seem to do what I want, your program creates a list of empty {} . If I want to create a named empty list I would type onetemp={} Then I could access onetemp and add data into it, how would I do that in your example?

POSTED BY: Paul Cleary

Hi,

something like this?

names = {"one", "bus", "tree", "townhouse"};
Map[Set[#, {}] &, ToExpression[# <> "temp"] & /@ names]

Best wishes, Marco

POSTED BY: Marco Thiel
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract