Attachments remain at their original size in Vanilla

Has anyone managed an esoTalk migration before? The developer has moved onto a new project and I'm looking to Vanilla as a platform with a future.

I get this error if I try porter on phpBB 3.0.12:

create index z_idx_pmgroup on z_pmgroup (subject, userids);Specified key was too long; max key  length is 1000 bytes

Error: (1024) Specified key was too long; max key length is 1000 bytes 

Any idea how to fix this?

Regards!
Stefan

Related: http://vanillaforums.org/discussion/20772/moving-from-bbpress-2-0

Hello,

Would it please be possible to add support for IPB 4.x so we can import all data from there? :)

Thank you.

Running the PHP script out of the box produced

Error: (2) touch(): Unable to create file vanilla2test.txt because Permission denied

Took me a bit to realize that I have to create a directory writable by the webserver, from which to run vanilla2export.php. I use a VPS with nginx and have MyBB in /var/www/MyBB. Nothing unusual.

There was a missing Discussion after an import using the vBulletin5 package.
After some investigation I found out that the node had a content type of class "Link".

That content type seems to have everything a Discussion needs. (ID, Title, Content)

We should check if it make sense to import other content types than 'Text'.

(I also double checked another dump and there were some nodes having the "Link" content type)

I just did an export with Porter 2.0.3 on a vBulletin 4.2.1 database and it will not allow me to import. It just keeps taking me back to the import page with a big red warning stating:

You must select a file to import.
Email is required.
Password is required.

I have tried three different administrator credentials from the original vBulletin forum and I get the same result...it does not import.

Confirm the vBulletin 3 exporter also works for 4. 95% sure it will work but it should be inspected closer.

Hey guys, just wanted to give you a heads up with the vanilla exporter.

I was exporting a site from 1.1.4 with the vanilla porter and it kept breaking. I took a look at the file and it seemed to be breaking on a comment that a member had left that said [code]<noscript>

from @wesbos

The polls are not displayed unless the discussion type is set to Poll.
The fix would be to check the column pollid for a value > 0.

Probably due to this PR: #94

'destpath' => array(
        'Define destination path for the export file.',
        'Sx' => '::',
        'Short' => 'd',
        'Default' => '.', <----
    ),

I've been looking around the docs for too many minutes for something that should be easily accessible:

Had to install and run the script to get that, so I've made a PR against the docs.

With MySQL extension being removed from PHP7 and the other MySQL compatible extensions not necessarily installed with PHP the need of an abstraction is now more than real.

See #95 for more information about PHP7 and DB abstraction

I notice that on this page of the Vanilla website that IP.Board is mentioned: http://vanillaforums.com/solutions/legacy-forum

Is that available with the VanillaPorter? Has development started on it at all?

I have a rather large forum (1179737 posts, 12950 members) that I'd like to convert to Vanilla, however, with the lack of IP.Board support in the importer, it looks like I may have to abandon the idea. Or maybe convert over just my members with a simple script I write.

I get the following error when trying to export my SQL using the tool

The topics table is missing the following column(s): topic_replies

any help on how to work around

I posted this issue on the vanilla forum already, and fixed it for my case, but was asked to post this issue here as well:

I ran the export import from an old SMF 1 forum to a fresh vanilla install a couple of times.
Both times, most of the personal messages from the old forum seemed to be missing when looking in the "inbox" of a user.
Looking in the database, almost all items are "deleted = 1", which seemed weird.

I've update all to deleted = 0 to check, but even then, only a few messages are showing and nothing newer then 2008 for some strange reason. So the deleted = 1 was not wrong after all.

After some database diving i found the following:
The problem is, all messages were imported properly into the GDN_conversationMessages table. But for some messages/conversations no entry was made into the GDN_conversations table, which binds different messages together in a discussion.

I've created a query to rectify this afther the import is run. This fixes this issue for me. It neatly sets amount of messages, first message id and last message id to the right settings.
Not all columns in the GDN_conversations table can be set, but those aren't really needed for it to work.

INSERT INTO vanilla_beta.GDN_Conversation(
   ConversationID,
  Subject,
  Contributors,
  FirstMessageID,
  LastMessageID,
  InsertUserID,
  DateInserted,
  DateUpdated,
  CountMessages
) SELECT 
    ConversationID, 
    null, 
    ' ', 
    (SELECT MessageID FROM GDN_ConversationMessage cm2 WHERE cm.ConversationID = cm2.ConversationID ORDER BY cm2.DateInserted ASC LIMIT 1), 
    (SELECT MessageID FROM GDN_ConversationMessage cm2 WHERE cm.ConversationID = cm2.ConversationID ORDER BY cm2.DateInserted DESC LIMIT 1), 
    InsertUserID, 
    DateInserted, 
    DateInserted, 
    (SELECT COUNT(*) FROM GDN_ConversationMessage cm2 WHERE cm.ConversationID = cm2.ConversationID) 
FROM GDN_ConversationMessage cm
WHERE ConversationID NOT IN (SELECT ConversationID FROM GDN_Conversation)
GROUP BY ConversationID

I know running this SQL doesn't solve the root of the problem, but since my client is more interested in making the deadline and keeping their users happy, i didn't debug the export/import

In a quoted post, I see this:

Dan Dascalescu' pid='2487' dateline='1365377895' said:

Source: https://forum.quantifiedself.com/thread-zeo-shutting-down-export-your-data?pid=2638#pid2638

Imported: http://vanilla.quantifiedselfforum.com/index.php?p=/discussion/comment/2638/#Comment_2638

Hi everybody.
I used the tool to export my Vbulletin 4.1.3 forum to Vanilla.
My database character charset is UTF-8 , but when exported, the csv file is utf-8 broken.
Do you know how to correctly write data into csv in UTF-8 format ?
Thank you very much.

I stumbled on a vBulletin forum that have comments embedded in comments.
The embedded comments are still imported but their DiscussionID is a CommentID.
This cause most of the inner comments to not show and it might cause some comments to show in a random discussion.

Prerequisite vanilla/vanilla#3906

From Todd:

Perhaps some of the db types can make use of the new table. I'd make a note to check on the next vBulletin import we do.

I have a vBulletin site (running 4.2.2 Patch Level 1) that I am trying to import into vanilla (2.1.15). I downloaded porter version 2.0.3 and did the vbulletin export.

When i try to do the import, I get the following error:

Row 1 doesn't contain data for all columns|Gdn_Database|Query|load data infile '/Users/lfolco/Sites/lb/local/vanilla/uploads/import/User.txt' into table GDN_zUser character set utf8 columns terminated by ',' optionally enclosed by '"' escaped by '' lines terminated by '\n' ignore 1 lines

this is what is in my user file:

UserID:int,Name:varchar(20),Email:varchar(200),Title:varchar(100),RankID:int,HourOffset:int,LastIPAddress:varchar(15),InviteUserID:int,InsertIPAddress:varchar(15),Password:varbinary(100),DateOfBirth:datetime,DateFirstVisit:datetime,DateLastActive:datetime,DateInserted:datetime,DateUpdated:datetime,Photo:varchar(255),Banned:tinyint,HashMethod:varchar(10)
"1","Admin","[email protected]","Administrator",1,"0","","0","","cdce1bf79f807370a1ade1749d64|~z/zY~eUIklQ}0^s!27O]tZ=8qfGe","0000-00-00","2014-07-15 08:18:05","2014-10-24 08:10:52","2014-07-15 08:18:05","2014-10-27 04:57:36",\N,"0","vbulletin"

Should return a clear error instead.

Compare the 3rd quote block, right before the "Hi Kate" at the end:

MyBB: https://forum.quantifiedself.com/thread-zeo-shutting-down-export-your-data?pid=2659#pid2659

Vanilla: http://vanilla.quantifiedselfforum.com/index.php?p=/discussion/comment/2659/#Comment_2659 - you can see "[/quote]"

Discourse: http://discourse.quantifiedselfforum.com/t/zeo-shutting-down-export-your-data/561/6 - it nests the quotes correctly, though it still gets the quoting line wrong (includes the PID and the date is left in seconds-since-epoch format)

It would be great to give users which options they want imported from the source forum. For example, when importing from IPB, I may only want to import the users. As it is, the only available options are to import the source's user groups as new Roles, or to just wipe out all of the Roles. There is no option to keep the default Vanilla Roles.

Can I remove it? I don't think we have a reason for it anymore.

Apparently not respecting 'deleted' flag (whatever that is)

http://vanillaforums.org/discussion/22060

Just moved a V1 forum to V2 and we've got a few deleted comments showing up. It appears that the export script doesn't check the deleted flag in the query to select the comments.

"The phrase table is missing the following column(s): fieldname"

http://vanillaforums.org/discussion/17627/

People are incorrectly using porter for upgrading from 2.0 -> 2.1, but it'd be nice if that actually worked without erasing their permissions.

The porter use the mysql extension which is removed in PHP7

So PDO_MYSQL or MySQLi?

The correct method to use should be Drupal and not Django.

Exporter should account for the timezone conversion to UTC & should convert any forum using timestamps to datetimes.

All packages should set thumbnails data for images in GDN_Media.
If there is no thumbnail we should take the original image as the thumbnail source and set an appropriate thumbWidth.

Is it possible, that you can support http://www.phorum.org/ for migration into Vanilla?

The project is quite well documented: http://www.phorum.org/docs/html/

Hello everyone,

I just try to migrate my IPB forum to Vanilla but when i try to import my exported file, I have this message :

There was an error while trying to create the Media table (The specified data type (StorageMethod) is not accepted for the MySQL database.).

Thx for your help!

Imported post: http://vanilla.quantifiedselfforum.com/index.php?p=/discussion/comment/2659/#Comment_2659

The title says it all!

There is a nice tool that seems to do a good job at fixing coding standards: http://cs.sensiolabs.org/
We can use the rules from PSR-2 that we wants, exclude those we do not and create our own custom rules for exceptions.

Convenience link: Vanilla coding standards

vanilla/vanilla#1351

copy/paste from vanilla/vanilla#1546

Hi people,

so I've got this wonderful DB/forum software combo that I've gotta transfer to Vanilla 2.x. MySQL 5.5.23-55 with PHPBB 2.0.6 (yeah, don't say anything, I know). I see no point in upgrading to PHPBB3 because I'll dump the whole mess that is the PHPBB platform and move to something more modern and with better integration with WordPress, namely Vanilla.

And let me apologize in advance I didn't want to open 4 separate issues for this. I've already forked, and will patch some of this functionality and submit a pull request so you guys can check it out and include it in some future version of Porter.

So anyway, let's start with the DB config.

Aka Part 1.

DB collation latin1_swedish_ci
DB charset from the SELECT CHARSET('some_field'); is utf8

It's a Percona server not MYSQL per se, and it's on a shared hosting. I can remote mysql into it, which I discovered as I'm writing this so that's one plus for getting the dumps without PHPMYADMIN.

Plain old conversion via the vanilla2export.php does transfer all data specified (but not all data required, but that'll be another issue), but the Croatian characters come out mangled. This is particularly troublesome with nicknames that have characters like šćđž in them.

If you force the script to read the character set with the lines below at the very top of ForumExport function in the phpbb2 class, you do effectively nothing in this example because the script gets the charset which is utf8 anyway.

      $CharacterSet = $Ex->GetCharacterSet(':_users.username');
      if($CharacterSet)
        $Ex->CharacterSet = $CharacterSet;

When a post with a [quote="Ašow"]...sometext...[/quote] in it transfers to Vanilla's post table and you're using the above patch, the actual quote comes out saying quote="A and that's it. That's a pretty big problem

So it's a hydra-like problem.

• the collation is wrong for the language used.
• the script doesn't connect to the DB in the default collation, it's UTF-8 or die
• the script, although it does utf8_encode as per proper PHP fashion because it doesn't check for collation of the table and checks only for charset (as far as I can see) it doesn't output the proper character set :/ Or better said, it outputs the right data but in the wrong character map.

But that's solvable (I hope) with some iconv and mysql magic, and isn't really needed in the script, for someone who has prior knowledge in PHP&MYSQL UTF-8 problems.

Part 2 of the problem is the following line of code.

   // Skip pending memberships
      $Ex->ExportTable('UserRole', 'select user_id, group_id from :_users
         union
         select user_id, group_id from :_user_group where user_pending = 0', $UserRole_Map);

That's just plain wrong. There is no group_id in phpbb_users table in phpbb2, at least on the version I'm currently trying to transfer.

Either it should say:

   // Skip pending memberships
      $Ex->ExportTable('UserRole', 
'select user_id, group_id  from :_user_group where user_pending = 0', $UserRole_Map); 

Or it should do a join on three tables like:

 $Ex->ExportTable('UserRole',
      'select u.user_id, g.group_id from :_users u 
join :_user_group ug on ug.user_id = u.user_id 
right join :_groups g on ug.group_id = g.group_id 
where ug.user_pending = 0',
        $UserRole_Map); 

Both examples have the exactly same effect. That'll be upped later on my fork, so I can submit a pull request for it.

Part 3 of the problem.

PHPBB2 forums (don't know for ver 3.) don't require assigning users to some default group. When Vanilla Porter does it's magic, large number of users (most, except for those assigned to some group like say moderators, or private forum groups), don't get assigned to any group.

From this stems two problems, users don't get assigned to some default group, and Administrators (if there are more than 1, which is normal with larger forums) except for the one specified in the import procedure, are not assigned as Administrators.

This is as I gather from the import script (which complains if users are not assigned to a group) required. Wouldn't it be better then to assign by default anons/guests to group Guest and users to group Users no matter what.

This also isn't doable without going into SQL for existing users. It's not a problem for someone who does this for a living, but for some rookie... Yeah, it could be :) It can be done with some button in the Vanilla admin dashboard, assign all existing users to ... Similar functionality exists for new users, but not for existing ones.

Part 4

No bans are imported. Either the banlist (IP, Mail, Name), or is the user banned into the user table. Is there a reason for this? I really need this functionality, and I'll implement it myself if I have to, but I wouldn't want to get, oh no, the way you that isn't compatible with .

As I've said I'll implement it myself because I need it, but can I at least have some guidelines on how the banning works. What has priority (the user table or the ban list)? Is the banlist (I'm thinking ahead right now) only in effect during the user creation and the user table field for normal operation or how it's done if not that way :)

Part 5

Just kidding, that's it (for now) :P
I'll post patches/guides for anything I can, and I hope I'll get some feedback on this TL;DR issue... Vanilla features OOB are amazing, and probably with plugins just get better and better, but first I need to get the site up and running with data displaying as intended.

Right I have converted the mybb database I got the gz file iv gone to the importer imported it

I get Data file directory (/***/**/public_html/vanilla/uploads/import/import) is not writable in red at top but its still running the import as I get process import file then loading bar there is a lot of data from old forum
and I no its takes time but is this correct?

There is a problem on line 403 of class.phpbb3.php that causes the export to fail for phpBB on MySQL 5.0.x. The problem is a bug in MySQL 5.0.x where a space between "concat" and the parenthesis was rejected as an SQL error. Therefore, the line:

concat ('FileUpload/', a.physical_filename) as Path,

should be changed to:

concat('FileUpload/', a.physical_filename) as Path,

Thanks!

See #110
That PR adds the QnA columns into vanillaStructure. The package should add the column dynamically to the structure using the ExportModel::structures() method.

We should restore the vanillaStructure to it's "vanilla/default" state and make the packages add the column themselves.

Add an option to run phpbb.extensions.php for the phpbb package.

Apparantly a null as filethumb missing in line 12451

select a.attachmentid, a.filename, right(a.filename, instr(reverse(a.filename), '.')) as extension , null as hash, a.filehash, a.userid,
           'local' as StorageMethod,
           'discussion' as ForeignTable,
           t.threadid as ForeignID,
           FROM_UNIXTIME(a.dateline) as DateInserted,
           '1' as height,
           '1' as width,
           null as filethumb
        from vb_thread t
           left join vb_attachment a ON a.postid = t.firstpostid
        where a.attachmentid > 0

        union all

        select a.attachmentid, a.filename, right(a.filename, instr(reverse(a.filename), '.')) as extension , null as hash, a.filehash, a.userid,
           'local' as StorageMethod,
           'comment' as ForeignTable,
           a.postid as ForeignID,
           FROM_UNIXTIME(a.dateline) as DateInserted,
           '1' as height,
           '1' as width
        from vb_post p
           inner join vb_thread t ON p.threadid = t.threadid
           left join vb_attachment a ON a.postid = p.postid
        where p.postid <> t.firstpostid and  a.attachmentid > 0
        ;The used SELECT statements have a different number of columns

Error: (1024) The used SELECT statements have a different number of columns

Error: (8) unserialize(): Error at offset 11 of 191 bytes

Can we rework to support PHP 5.2 without a big effort? I think we moved to 5.3 more by accident than intentionally.

Blob exporting (avatars & attachments) should be added to vBulletin web interface.

Optional file path for attachment renaming should be added to vBulletin & phpBB interfaces. -> #84

when attempting to export phpbb3, i get these errors Warning: Cannot modify header information - headers already sent by (Website link)/vanilla2export.php on line 494
same errors for line 495 496 497 498 499 and lots of errors from 500

I have tried with using my phpbb database prefix "which by default is phpbb_" and without, same errors. My host name and username and password are correct.

Please support Woltlab Burning Board (e.g. wBB3 Lite and wBB3 and so on).