Life In 19x19
http://prod.lifein19x19.com/

Analysing move quality. Kind of, for 9x9 games (using pachi)
http://prod.lifein19x19.com/viewtopic.php?f=18&t=10359		 Page 1 of 1
Author:  RBerenguel [ Mon May 26, 2014 10:53 am ]
Post subject:  Analysing move quality. Kind of, for 9x9 games (using pachi)
Pachi (http://pachi.or.cz/) is an open source UCT go playing program. According to its authors, its strength is around 7d KGS for 9x9, 2d for 19x19. I am interested in the 9x9 part, since (as I said elsewhere) I try to improve my level of play, and if my computer can help me, well, I'll use it.

Pachi comes with a set of tools (in the folder /tools) and among them there is sgf-analyse.pl, which should give you an analysis of your own games. Problem is, I couldn't make it work and my knowledge of perl is just barely enough to understand what it does but not how to fix it.

So I rewrote sgf-analyse in awk (requires GNU awk for the co-process pipe operator)

Code:
BEGIN {
    total=4
    commands[1]=""
    move[1]=""
    engine="-e uct"
    plays="-t =1000"
    pachi_exec="./pachi"
    pachi=pachi_exec " " engine " " plays " -d 0 dynkomi=none"
    file="/tmp/gorate"
    system("rm " file)
    pachi_exec "  -a 2>&1 | head -n 2 | tail -n 1\n" | getline
    printf "Using " plays " plays and " engine " engine.\n" $0 "\n\n"
    printf "For each move evaluate the board and generate a next\nmove. Evaluating the board and generating a move give\ndifferent values, even for the same move.\n\nGenerating also /tmp/gorate with format:\n Pwin generated, Evaluate value | -1 , Exact match move? (i.e. like 0.55, -1,  1)\n\n"
}

NR<=4{
    commands[NR]=$0
}

NR==3{
    move[1]=$3
}
NR==4{
    move[2]=$3
}

# Will process output from sgf2gtp, repeating
(NR>4){
    total=total+1
    mvc=NR-3
    commands[NR]=$0
    #printf commands[NR]
    move[mvc]=toupper($3)
    tomove[mvc]=$2
    #printf "Move: " move[NR] "\n"
}

END {
    for(j=7;j<=total;j++){ #Starts at 5 and evaluates starting at j=Move to study+3
   mvc=j-3
   count=count?0:1
   color="unknown"
   steps=0
   while (color=="unknown"&&steps<3){
       steps=steps+1
       for (i=1;i<=j-1;i++){
#      printf "DEBUG: Issuing command #"i": " commands[i] "\n"
      printf commands[i] "\n" |& pachi
       }
#       printf "DEBUG: Issuing command: evaluate " tomove[mvc] "\n"
       printf "pachi-evaluate " tomove[mvc] "\n" |& pachi
#       printf "DEBUG: Issuing command: genmove " tomove[mvc] "\n"
       printf "genmove " tomove[mvc] "\n" |& pachi
#       printf "DEBUG: Issuing command: pachi-result\n"
       printf "pachi-result\n" |& pachi
#       printf "DEBUG: Issuing command: quit\n"
       printf "quit\n" |& pachi
       countreply=0
       score=""
       while(pachi |& getline >0){
#      printf "DEBUG: " $0 "->" $1 " " $2 " " $3 "\n"
      if ($2!=""){
          # format color move playouts pwin dynkomi for pachi-result
          color=$2
          genmove=$3
          pwin=$5
          if($4==""){
#         print "DEBUG: $4 is null: " $0 " " $1 " " $2
          }
          # For pachi-evaluate, move and pwin
          if(color==move[mvc]&&$3!=""){
#         printf "DEBUG: Match evaluate"
         score=genmove
          }
          if($1==move[mvc]){
#         printf "DEBUG: Match evaluate-2"
         score=$2
          }      
      }
       }
       close(pachi)
   }
   printf pwin ", " >> file
   if(score=="")
       printf "-1, " >> file
   else
       printf score ", " >> file
   if(move[mvc]==genmove)
       printf " 1\n" >> file #Correct hit
   else
       printf " 0\n" >> file #Wrong hit
   if(score!=""){
       printf mvc ": Original " tomove[mvc] " move: " move[mvc] " has possible P(win) " score ". Suggested " color " move: " genmove " P(win): " pwin "\n"
   }else{
       printf mvc ": Original " tomove[mvc] " move: " move[mvc] ". Suggested " color " move: " genmove " pwin: " pwin "\n"
   }

    }

    close(pachi)
}



Here's a run of it against one of Go Seigen - Miyamoto Naoki's 9x9 games. You'll see some "unknown" in there. Not sure why it happens, one of these spurious things that just make you wonder if programming is made for sane minds.

Code:
tools/sgf2gtp.pl < ~/Downloads/Pro9x9-1.sgf | gawk -f ~/Dropbox/Codi/awk/pachi-eval.awk
Using -t =1000 plays and -e uct engine.
Pachi version 10.00 (Satsugen)

For each move evaluate the board and generate a next
move. Evaluating the board and generating a move give
different values, even for the same move.

Generating also /tmp/gorate with format:
Pwin generated, Evaluate value | -1 , Exact match move? (i.e. like 0.55, -1,  1)

4: Original B move: F4 has possible P(win) 0.520. Suggested black move: D7 P(win): 0.60
5: Original W move: G4 has possible P(win) 0.415. Suggested white move: G4 P(win): 0.41
6: Original B move: G3 has possible P(win) 0.528. Suggested black move: G3 P(win): 0.59
7: Original W move: F3 has possible P(win) 0.446. Suggested white move: F3 P(win): 0.51
8: Original B move: E4 has possible P(win) 0.478. Suggested black move: E4 P(win): 0.58
9: Original W move: G2 has possible P(win) 0.430. Suggested white move: H3 P(win): 0.47
10: Original B move: H3 has possible P(win) 0.569. Suggested black move: H3 P(win): 0.56
11: Original W move: H2 has possible P(win) 0.389. Suggested white move: H2 P(win): 0.50
12: Original B move: E3 has possible P(win) 0.585. Suggested black move: H4 P(win): 0.57
13: Original W move: F2 has possible P(win) 0.408. Suggested white move: F2 P(win): 0.48
14: Original B move: G5 has possible P(win) 0.559. Suggested black move: E7 P(win): 0.55
15: Original W move: H4 has possible P(win) 0.400. Suggested white move: H4 P(win): 0.43
16: Original B move: H5 has possible P(win) 0.530. Suggested black move: D7 P(win): 0.61
17: Original W move: J3 has possible P(win) 0.380. Suggested white move: J3 P(win): 0.44
18: Original B move: F7 has possible P(win) 0.527. Suggested black move: D8 P(win): 0.58
19: Original W move: G6 has possible P(win) 0.390. Suggested white move: G6 P(win): 0.43
20: Original B move: G7 has possible P(win) 0.549. Suggested black move: E6 P(win): 0.59
21: Original W move: H6 has possible P(win) 0.389. Suggested white move: H6 P(win): 0.42
22: Original B move: E6 has possible P(win) 0.519. Suggested black move: E7 P(win): 0.62
23: Original W move: B5 has possible P(win) 0.417. Suggested white move: B4 P(win): 0.44
24: Original B move: H7 has possible P(win) 0.566. Suggested black move: C3 P(win): 0.57
25: Original W move: J5 has possible P(win) 0.398. Suggested white move: J5 P(win): 0.45
26: Original B move: B7 has possible P(win) 0.504. Suggested black move: B4 P(win): 0.61
27: Original W move: C3 has possible P(win) 0.380. Suggested white move: C3 P(win): 0.46
28: Original B move: C4 has possible P(win) 0.565. Suggested black move: C4 P(win): 0.60
29: Original W move: B4 has possible P(win) 0.388. Suggested white move: B4 P(win): 0.46
30: Original B move: C2 has possible P(win) 0.529. Suggested black move: D2 P(win): 0.55
31: Original W move: C7 has possible P(win) 0.442. Suggested white move: C7 P(win): 0.49
32: Original B move: C8 has possible P(win) 0.509. Suggested black move: C6 P(win): 0.57
33: Original W move: D3 has possible P(win) 0.471. Suggested white move: B3 P(win): 0.54
34: Original B move: D2 has possible P(win) 0.454. Suggested black move: D2 P(win): 0.47
35: Original W move: B2 has possible P(win) 0.492. Suggested white move: B2 P(win): 0.59
36: Original B move: E2 has possible P(win) 0.513. Suggested black move: E2 P(win): 0.54
37: Original W move: D7 has possible P(win) 0.409. Suggested white move: C6 P(win): 0.51
38: Original B move: B6 has possible P(win) 0.628. Suggested black move: C6 P(win): 0.63
39: Original W move: E7 has possible P(win) 0.242. Suggested white move: C6 P(win): 0.37
40: Original B move: C6 has possible P(win) 0.792. Suggested black move: D6 P(win): 0.80
41: Original W move: F8 has possible P(win) 0.267. Suggested white move: F8 P(win): 0.40
42: Original B move: G8 has possible P(win) 0.719. Suggested black move: G8 P(win): 0.74
43: Original W move: B8 has possible P(win) 0.186. Suggested white move: H9 P(win): 0.30
44: Original B move: E8 has possible P(win) 0.520. Suggested black move: A8 P(win): 0.83
45: Original W move: D8 has possible P(win) 0.328. Suggested white move: D8 P(win): 0.48
46: Original B move: F9 has possible P(win) 0.544. Suggested black move: F9 P(win): 0.55
47: Original W move: C9 has possible P(win) 0.478. Suggested white move: A8 P(win): 0.48
48: Original B move: E9 has possible P(win) 0.492. Suggested black move: A8 P(win): 0.61
49: Original W move: D9 has possible P(win) 0.356. Suggested white move: A8 P(win): 0.49
50: Original B move: D4 has possible P(win) 0.447. Suggested black move: A8 P(win): 0.63
51: Original W move: B3 has possible P(win) 0.351. Suggested white move: B3 P(win): 0.43
52: Original B move: B1 has possible P(win) 0.568. Suggested black move: A8 P(win): 0.66
53: Original W move: A8 has possible P(win) 0.280. Suggested white move: D6 P(win): 0.54
54: Original B move: A2 has possible P(win) 0.493. Suggested black move: A2 P(win): 0.71
55: Original W move: C1 has possible P(win) 0.284. Suggested white move: A6 P(win): 0.31
56: Original B move: D1 has possible P(win) 0.730. Suggested black move: D1 P(win): 0.71
57: Original W move: A3 has possible P(win) 0.246. Suggested white move: A7 P(win): 0.38
58: Original B move: A5 has possible P(win) 0.668. Suggested black move: A5 P(win): 0.78
59: Original W move: A7 has possible P(win) 0.173. Suggested white move: H9 P(win): 0.38
60: Original B move: A6 has possible P(win) 0.618. Suggested black move: C5 P(win): 0.85
61: Original W move: H9 has possible P(win) 0.317. Suggested white move: H9 P(win): 0.35
62: Original B move: H8 has possible P(win) 0.673. Suggested black move: G9 P(win): 0.72
63: Original W move: F1 has possible P(win) 0.154. Suggested white move: A1 P(win): 0.30
64: Original B move: C5 has possible P(win) 0.859. Suggested black move: C5 P(win): 0.81
65: Original W move: A1 has possible P(win) 0.101. Suggested unknown move: pachi-result P(win):
66: Original B move: C1 has possible P(win) 0.888. Suggested black move: J7 P(win): 0.89
67: Original W move: E1 has possible P(win) 0.096. Suggested white move: J7 P(win): 0.10
68: Original B move: A2 has possible P(win) 0.827. Suggested black move: G9 P(win): 0.87
69: Original W move: J7 has possible P(win) 0.108. Suggested white move: J7 P(win): 0.21
70: Original B move: J8 has possible P(win) 0.917. Suggested black move: J8 P(win): 0.89
71: Original W move: A1 has possible P(win) 0.038. Suggested unknown move: pachi-result P(win):
72: Original B move: B9 has possible P(win) 0.854. Suggested black move: G9 P(win): 0.97
73: Original W move: A9 has possible P(win) 0.029. Suggested white move: A9 P(win): 0.16
74: Original B move: A2 has possible P(win) 0.870. Suggested black move: G9 P(win): 0.98
75: Original W move: G9 has possible P(win) 0.065. Suggested unknown move: pachi-result P(win):
76: Original B move: J9 has possible P(win) 0.912. Suggested black move: J9 P(win): 0.92
77: Original W move: A1 has possible P(win) 0.023. Suggested unknown move: pachi-result P(win):
78: Original B move: G9 has possible P(win) 0.992. Suggested black move: G9 P(win): 0.97
79: Original W move: J6. Suggested white move: D6 pwin: 0.01


And here's a plot of the win-rates of real moves (according to pachi-evaluate) and of the generated game by pachi (according to pachi-result):

Attachment:
Screen Shot 2014-05-26 at 19.38.56.jpg
Screen Shot 2014-05-26 at 19.38.56.jpg [ 116.61 KiB | Viewed 7947 times ]


I used gnuplot for the plot, here's the plotting command:

Code:
plot "/tmp/gorate" every 2::1 using 0:2 with lines title "Black WR-move", '' every 2::2 using 0:2 with lines title "White WR-move", '' every 2::1 using 0:1 with lines title "Black WR-gen", '' every 2::2 using 0:1 with lines title "White WR-gen"


It would be cooler hith Highcharts.js but I was too lazy for it (at least now.) So, almost done with one of the pieces of the 9x9 analysis suite (the other one I have kind of done is an opening move % classifier.)
Author:  oca [ Tue May 27, 2014 1:28 am ]
Post subject:  Re: Analysing move quality. Kind of, for 9x9 games (using pa
oh... pachi can be remoted via GTP... that seems nice... thx for your post ! I will give pachi a second look :D
Author:  RBerenguel [ Tue May 27, 2014 1:30 am ]
Post subject:  Re: Analysing move quality. Kind of, for 9x9 games (using pa
oca wrote:
oh... pachi can be remoted via GTP... that seems nice... thx for your post ! I will give pachi a second look :D


Yup, it can and it's "relatively" straightforward. There's also a "shape finding" thing about it (don't remember exactly the name, but it outputs a lot of shape/pattern information that can then be cross-referenced... I think I interacted with it via the Acme text editor, so it was relatively easy to do) which is what I told you in the thread about naming moves, it's more or less what the gostyle app used.

I just updated the script, now it generates another output file that can then be parsed into an SGF (with another awk script) so the result is more "visual"
Author:  RBerenguel [ Tue May 27, 2014 1:38 am ]
Post subject:  Re: Analysing move quality. Kind of, for 9x9 games (using pa
Since editing the first post is a little cumbersome (it's very long) here's the updated script. Now it generates another file goratemv with has the format originalmove, eval-value, suggestedmove, pachi-result, coincide:

Code:
BEGIN {
    total=4
    commands[1]=""
    move[1]=""
    engine="-e uct"
    plays="-t =3000"
    pachi_exec="./pachi"
    pachi=pachi_exec " " engine " " plays " -d 0 dynkomi=none"
    file="/tmp/gorate"
    file2="/tmp/goratemv"
    system("rm " file)
    system("rm " file2)
    pachi_exec "  -a 2>&1 | head -n 2 | tail -n 1\n" | getline
    printf "Using " plays " plays and " engine " engine.\n" $0 "\n\n"
    printf "For each move evaluate the board and generate a next\nmove. Evaluating the board and generating a move give\ndifferent values, even for the same move.\n\nGenerating also /tmp/gorate with format:\n Pwin generated, Evaluate value | -1 , Exact match move? (i.e. like 0.55, -1,  1)\n\n"
}

NR<=4{
    commands[NR]=$0
}

NR==3{
    move[1]=$3
}
NR==4{
    move[2]=$3
}

# Will process output from sgf2gtp, repeating
(NR>4){
    total=total+1
    mvc=NR-3
    commands[NR]=$0
    #printf commands[NR]
    move[mvc]=toupper($3)
    tomove[mvc]=$2
    #printf "Move: " move[NR] "\n"
}

END {
    for(i=1;i<4;i++){
   printf move[i] ", , , , -1" >> file2
    }
    for(j=7;j<=total;j++){ #Starts at 5 and evaluates starting at j=Move to study+3
   mvc=j-3
   count=count?0:1
   color="unknown"
   steps=0
   while (color=="unknown"&&steps<5){
       steps=steps+1
       for (i=1;i<=j-1;i++){
#      printf "DEBUG: Issuing command #"i": " commands[i] "\n"
      printf commands[i] "\n" |& pachi
       }
#       printf "DEBUG: Issuing command: evaluate " tomove[mvc] "\n"
       printf "pachi-evaluate " tomove[mvc] "\n" |& pachi
#       printf "DEBUG: Issuing command: genmove " tomove[mvc] "\n"
       printf "genmove " tomove[mvc] "\n" |& pachi
#       printf "DEBUG: Issuing command: pachi-result\n"
       printf "pachi-result\n" |& pachi
#       printf "DEBUG: Issuing command: quit\n"
       printf "quit\n" |& pachi
       countreply=0
       score=""
       while(pachi |& getline >0){
#      printf "DEBUG: " $0 "->" $1 " " $2 " " $3 "\n"
      if ($2!=""){
          # format color move playouts pwin dynkomi for pachi-result
          color=$2
          genmove=$3
          pwin=$5
          if($4==""){
#         print "DEBUG: $4 is null: " $0 " " $1 " " $2
          }
          # For pachi-evaluate, move and pwin
          if(color==move[mvc]&&$3!=""){
#         printf "DEBUG: Match evaluate"
         score=genmove
          }
          if($1==move[mvc]){
#         printf "DEBUG: Match evaluate-2"
         score=$2
          }      
      }
       }
       close(pachi)
   }
   printf pwin ", " >> file
   printf move[mvc] ", " >> file2
   if(score==""){
       printf "-1, " >> file
       printf "-1, " >> file2
   }
   else{
       printf score ", " >> file
       printf score ", " >> file2
   }
   printf genmove ", " pwin ", ">> file2
   if(move[mvc]==genmove){
       printf " 1\n" >> file #Correct hit
       printf " 1\n" >> file2 #Correct hit
   }
   else{
       printf " 0\n" >> file #Wrong hit
       printf " 0\n" >> file2 #Wrong hit
   }
   if(score!=""){
       printf mvc ": Original " tomove[mvc] " move: " move[mvc] " has possible P(win) " score ". Suggested " color " move: " genmove " P(win): " pwin "\n"
   }else{
       printf mvc ": Original " tomove[mvc] " move: " move[mvc] ". Suggested " color " move: " genmove " pwin: " pwin "\n"
   }

    }

    close(pachi)
}



And this script can take a file like that and convert it to an SGF you can follow. For now, komi, rules and players are missing, since there is no way to pass directly this data from the sgf2gtp script through the awk files. Next step (for me, at least) about these scripts is wrapping them between (probably) a simple shell script that can create all necessary files and information

Code:
# (;GM[1]FF[4]CA[UTF-8]AP[CGoban:3]ST[2]
# RU[Japanese]SZ[9]KM[0.00]
# PW[test]PB[test]
# ;B[gc]CR[cc]C[Main move with circle for other considered move]
# ;W[gg]CR[cg]C[Main line follows with next move and letter])

BEGIN {
    printf "(;GM[1]FF[4]CA[UTF-8]AP[yhdeksan]SZ[9]"
    printf "RU[Japanese]KM[0.00]" # Fix later
    printf "PW[test]PB[test]\n" # Fix later
    color="W"
    FS=", "
    split("IHGFEDCBA", letters, "")
}

{
    color=color=="W"?"B":"W"
    split($1, gtpmove, "")
    failsafe=gtpmove[1]=="J"?"I":gtpmove[1]
    move= failsafe letters[gtpmove[2]]
    printf ";" color "[" tolower(move) "]"
    if($5!="-1"){
   if (!$5) {
       split($3, gtpmove2, "")
       failsafe=gtpmove2[1]=="J"?"I":gtpmove2[1]
       genmove= failsafe letters[gtpmove2[2]]
      
       printf "CR[" tolower(genmove) "]"
       comment="Main line move: " $2 "\n\nSuggested move: " $4
   }else{
       comment="Main line move coincides with suggestion: " $4
   }
   
   printf "C[" comment "]\n"
    }
}

END {
    printf ")"
}


Attached a sample from a very lousy 9x9 game I lost in OGS recently (yeah, I played pretty horribly, didn't need a computer to tell me :D)

Edit: actually, better inlining it!



Attachments:
test.sgf [2.46 KiB]
Downloaded 445 times
Author:  oca [ Tue May 27, 2014 3:08 am ]
Post subject:  Re: Analysing move quality. Kind of, for 9x9 games (using pa
I really, really like that idea... :clap:

Side note: pachi seems quite easy to remote from node.js
Code:
var spawn = require('child_process').spawn;
var child = spawn("..\\bin\\pachi.exe ",["-e","uct","-t=2000"]);
var resp = [];
var current = 0;

child.stdout.on('data', function (data) {
    resp.push(data.toString().trim());
   onResponse(cmds[current],resp[current]);
   current ++;
   if (current == cmds.length) {
      child.stdin.end();
   }
});


function onResponse (req, resp) {
   console.log("req : "+req+" -> response : "+resp)
}

var cmds =  [
   "boardsize 9",
   "clear_board",
   "komi 7.5",
   "genmove black",
   "genmove white",
   "genmove black"
]

cmds.forEach(function(cmd) {
   child.stdin.write(cmd+"\n"); // execute the command
});


D:\my\go\pachi\oca>node test.js
req : boardsize 9 -> response : =
req : clear_board -> response : =
req : komi 7.5 -> response : =
req : genmove black -> response : = G7
req : genmove white -> response : = E7
req : genmove black -> response : = E5

D:\my\go\pachi\oca>
Author:  daal [ Tue May 27, 2014 4:52 am ]
Post subject:  Re: Analysing move quality. Kind of, for 9x9 games (using pa
What's up with move 6? The main line move and the suggested move have different values, but they are just mirrored. Kind of funny that pachi considers its own move better. :)
Author:  RBerenguel [ Tue May 27, 2014 5:51 am ]
Post subject:  Re: Analysing move quality. Kind of, for 9x9 games (using pa
daal wrote:
What's up with move 6? The main line move and the suggested move have different values, but they are just mirrored. Kind of funny that pachi considers its own move better. :)


The values for "eval" and "generated" almost never coincide, even for the same move. Lovely Monte Carlo stuff :D
Author:  RBerenguel [ Tue May 27, 2014 10:07 am ]
Post subject:  Re: Analysing move quality. Kind of, for 9x9 games (using pa
This is not as good as I want, yet. I was going to create a neat Highcharts visualisation... Then I realised Highcharts is only free for NC use, and I need to use something similar to this for my job (similar=dynamic graphs from data with interactivity....) So... rolled up my sleeves and did some quick hack (very quick, very hackish) in D3, but I'm pretty much still a D3 newbie (I was somewhat proficient like 2 years ago, but then I discovered Highcharts and never looked back... until now, damn :/)

Animated gif follows between the hide tags (since it's quite wide.) Uses jgoboard and D3. Shows evals of moves, not suggestions. I need to think exactly WHAT I want to plot there. The idea in any case is having the plot of values and the game and be able to move around.

Sooner or later, this will include a beautiful button "pattern" that will look for the current board situation among all 9x9 games I have and let me check them from just within my browser.

Attachment:
test.gif
test.gif [ 819.98 KiB | Viewed 7807 times ]
Author:  Satorian [ Thu May 29, 2014 5:26 am ]
Post subject:  Re: Analysing move quality. Kind of, for 9x9 games (using pa
I really like the SGF export idea. Would be brilliant to take a 19x19 game, leave Pachi running over night to give it 8-10 hours to analyze, and then review the results per move.

Hope you keep this project going!
Author:  RBerenguel [ Sat May 31, 2014 4:19 am ]
Post subject:  Re: Analysing move quality. Kind of, for 9x9 games (using pa
Satorian wrote:
I really like the SGF export idea. Would be brilliant to take a 19x19 game, leave Pachi running over night to give it 8-10 hours to analyze, and then review the results per move.

Hope you keep this project going!


Supposedly pachi is not strong enough for it to be really, really meaningful. Of course, you can always rise the playout value to huge numbers and just let it smash the game tree, but for 9x9 it would already be overkill expecting it to be stronger than 5-7d amateur, much, much less for 19x19, where the playout time also increases by a lot.

There's also the playstyle: pachi (or any other MCTS-based engine) shines in tactical situations. In opening and endgame it can blunder greatly (or at least not follow standard patterns and just play its own, weird game.) So for a 19x19 game I'd restrict it for moves 50+, at the very least.

For 9x9 it's not that significant, after move 4-5 the game is already tactical.
Page 1 of 1 All times are UTC - 8 hours [ DST ]
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/