Site updates and future roadmap.
Replies: 0   Views: 1785  Subscribers: 1

Posted by reece · 12-11-2012 - 01:50

Edited by reece · 12-11-2012 - 05:34
You may or may not have noticed that i have updated the site and i wanted to share with you some of the changes that have been made.

Heres what has changed:
  1. Re-wrote the BBCode engine.­
  2. ­
  3. Added new tags to BBCode engine.­
  4. ­
  5. Added a new Bundle ­MenuBundle­
  6. ­
  7. Fixed numerous bugs.­
  8. ­
  9. CCDNForumModeratorBundle is now history, all forum moderation features have been merged into the forum AdminBundle.­
  10. ­
  11. Added footer links.­
  12. ­
  13. Added the necessary legal documents for terms and conditions, copyright notice, disclaimer and lastly the privacy policy which you can find at the bottom of every page.­
  14. ­
  15. When registering you must now view the terms and conditions page first and accept the terms, registration is prevented when the checkbox is not selected to enforce acceptance of the terms.­
  16. ­
  17. The login page now has the register link on it.­
  18. ­
  19. Twig variables from configuration no longer use the Param extension as it could be a potential security risk, instead, twig globals are used. Variables are added to the twig globals via the bundles boot method.­
  20. ­
  21. Corrected some incorrect PHPDoc tags throughout the bundles that never got updated as things changed.­
  22. ­
  23. Fixed an issue where by some form themes did not add a space between css classes when merging attributes that caused misaligned form elements.­
  24. ­
  25. Added a cancel button to most forms now to return you to the relevant parent page that most likely lead you to that page. (Does not use browser history, so topic editing for example would naturally take you back to the topic if you clicked cancel as that is what would naturally come before that page).­
  26. ­
  27. Creating or replying to a topic now automatically subscribes you to the topic, you can find your subscriptions on the subscriptions button found on the dashboard or in the sidebar.­
  28. ­
  29. ForumBundle and MessageBundle now uses flood protection to prevent excessive posting in short time spaces to help protect the site from spam.­
  30. ­
  31. Blacklisted the attachment download/thumbnail links from the route referer, which prevents you from being redirected there when logging back in. Otherwise when you log back in the file downloads and you are left on the login screen giving you the impression that you never successfully logged in (even though you were).­
  32. ­
  33. The registration and login links are now added to the dashboard for easier navigation.­
  34. ­
I recommend reading the legal documents seeing as they are now available. ­


The BBCode engine was completely re-written with both memory and performance in mind. Initially this was necessary due to changes in PHP that meant that pass-by-reference was no longer allowed. Having needed to fix this it presented numerous new issues as a result. In the end it was necessary to rethink the entire architecture and re-write the engine. It is still a lexing engine however i managed to heavily optimise the engine by firstly doing some preprocessing on the lexeme_table that should only occur once in a single page load. This lexeme_table preprocessing does several things: ­
  1. Adds missing properties to the lexeme table that don't need to be added manually.­
  2. ­
  3. Uses white and black-listing properties to specify what tags can be nested inside each other. This is preprocessed before first usage of the engine per request. This has at least 2 main benefits, the first being that now we can enforce good html in our posts and secondly it ensures that the page flows nicely by not presenting ugly looking pages with oddly nested content. The last benefit is in allowing us to use the code block without its content being rendered by the engine.­
  4. ­
The lexing engine is now much more efficient, as it now uses a scan-ahead function which helps to prevent tags not being rendered because of a rogue closing tag that cannot be paired up with an opening tag. Previously what would happen is when lexing is taking place, each new tag found would result in a recursion of the lexer and when a closing tag is found matching the opening tag we return from the recursion (until we reach the root node of this tree structure). Unfortunately when it found a tag that did not match the parent in the scope of the nested branch the recursive function was looking at, it would return early to look for it in the parent node, and this would happen until all possibilities were exhausted. This was bad because it meant that tags that were still open before this rogue closing would not be closed and thus were never marked as valid for the parser. As a result the page would show the tags un-rendered. This results in a higher rate of un-rendered tags than are necessary. The solution was to use scan-ahead which checks to see if the tag has a closing tag before hand, then if it does not then when we find a rogue closing tag we know not to worry about it as it has less importance. This results in much more accurate rendering of tags and therefor better looking posts. Both the lexer and parser now no longer throw around references to the lexeme table and other important objects as it turns out passing things by reference all the time is not the best idea, specially when you are doing recursion. We store what we can in the class scope now as references can actually impact performance negatively in PHP due to the nature of its pre-optimisations and symbol-aliasing table for variables and objects. I lastly removed some of the more unnecessary emoticons from the lexeme_table as they just mean its more lexemes to test against, and secondly more graphics on a page means more content to load from the server or CDN. Ultimately the emoticons that were disabled were animated or excessively large in width or height so it is not much of a loss. The new tags available are lists, Ordered lists, Unordered lists and H1 through to H3 tags. ­It is important to note that the BB tags now require tag properties to be quoted, e.g:­ ­
The BBCodes CODE tag now uses googles code-prettify for syntax highlighting instead of lib-geshi. The benefit of which is that we can now offload the highlighting work to the client browser rather than on the server which helps improve performance a little bit. I might however swap out google code-prettify for Highlight.js but i am undecided as of yet, however one of the benefits of highlight.js is more accurate highlighting as it takes syntax into account with a built in lexing engine of its own, google code prettify on the other hand does not care much for syntax but instead looks for specific keywords and patterns. The downside of highlight.js is that it does not support line numbers out of the box, and may need a little work to add support for that. ­


The menu bundle is a bit of an experiment in decoupling various bundles common user interface elements. For example, buttons and links some of which are shown in groups may be provided by different bundles. The menu bundle allows individual bundles to subscribe to this service via an object that you can tag in its service definition. Tagged services will be picked up by the menu bundle and common links and chunks of html can be integrated together in a node tree structure. The menu builder implements the fluent interface which allows nestable objects which are returned according to the method called. All node types inherit from a common base node which implements an array type. You can use this to build menus, drop downs and groups of other logical tidbits. It is undecided yet wether the approach i have taken is the right one. I did look at the KnpMenuBundle but for some reason did not like it. I think it was due to the flexibility of being able to pass specific attributes to individual nodes. I will confess i was inspired by the concept though when i wrote this bundle. It is as of yet completely undocumented, as i am not even sure if this is the right way to go about it. Also, it only uses one group renderer at the moment (the twitter bootstrap renderer), i hope to add a compiler pass to this to allow you to specify your own renderers, or individual node renderers for pre-existing groups of renderers. One of the things that crossed my mind was wether this is overkill. Another approach i could have used (and may possibly switch to) is to use only an array node type here, rather than overly specific node types. So that the data is more or less just a pure array and then specify the renderer for the given array slice you want to call. One of the difficulties i was having though was figuring out how to keep the content very general while still allowing things such as translations and links etc to content. If the content is just a pure array, then knowing when to make content a link, and when to make it pure text becomes a bit muddy. So i will keep the current approach until i can figure out a better solution. The main goal here is decoupling the bundles, this is at least a start to that front. ­

Current Issues.­

issues that i am currently aware of and am actively working to resolve: ­
  • The login page with the registration text often does not like narrower browsing windows, if the window is too narrow, the login form will fall below the registration welcome notice. A more scalable layout for this page is needed to make the registration text rescale accordingly to accommodate the login form.­
  • ­
  • Some smilies seem to get through the black list filtering on the bb-code engine. It is not yet known why this is.­
  • ­
  • MenuBundle presents content in a semi-random order, usually dependant on order of the menu builders that were found in the service container. Simple solution is to add a priority attribute to nodes to allow prioritised ordering of nodes.­
  • ­
  • Forum topic reply transcript does not always show the authors username.­
  • ­
  • Flash messages created by FOSUserBundle are not translated because flash messages are rendered via a separate twig file, however all CCDN flash messages are translated in the controller before hand.­
  • ­
We can add to the list as we notice issues or post them to the appropriate github issue tracker. ­

Future RoadMap.­

The future road map for the site will be a focus on 2 main things, the first being further decoupling the logic of each bundle to a degree where there is little to no coupling left at all and the second being getting a head start on the new Article bundle for the site. I have a few ideas already for decoupling the bundles. The MenuBundle is a great start as now the common bundle no longer has a dependency on the message or profile bundles (which is a good thing). However we now have the issue that the CommonBundle is now dependant on the MenuBundle. So i am wondering wether i need to add a check for wether the menu bundle is present or not. It may be a non-issue at this point. One of the thoughts i keep having about decoupling the bundles is to implement a provider pattern to allow devs to choose what profile bundles/routes etc to use for each link, same for other things such as the message bundle and attachment bundle. Is a provider pattern the right way to go here? Thoughts on decoupling the bundles for the future roadmap would be greatly appreciated. The other concern of mine now is that the html is getting a bit bloated with inline styles, more css classes are needed and less inline styles. Twitter Bootstrap has helped in that regard quite nicely, but it does lack the necessary css classes for layout aspects such as fixed-width-fluid-width 2 pane layouts etc. As far as i am aware, it only has fixed width layouts, or layouts with 2 or more columns that scale together. It is kind of tricky getting a layout with 1 column fixed, the other fluid. I am wondering if there are any neat CSS tricks that can be used to allow me to reuse the layout CSS and have the CSS class pull some numeric value from the html tags. I swear i have seen something tricky like that done before somewhere, and would allow more reusable layout classes in the common.css. Another problem is that the current layout does not work too nicely with mobile browsers such as Safari for the iPad/iPhone or Chrome for Android. A new layout will be needed to combat the aforementioned issues. The layout should be fairly similar to the one already used, featuring a minamalistic approach while at the same time showing a clear separation of concerns. It needs to be able to look nice on the desktop browser and yet be usable on mobile devices. The article bundle lastly has little done in the way of design, and is mostly on paper at this point. I am open to discussion on the issue before we get started though one of my thoughts is that it should be a versioned collaborative article base, much like a wiki with a viewable history backlog that you can browse through to see changes. More over, as the nature of the article system is for teaching material related to electronic and technical topics, it should be able to have a means to choose a version for its technology. For example, if we had an article on the HTTP specification, a drop-down could be provided at the top for HTTP1.0 and HTTP1.1, this would be independent of the version history for each article. Which somewhat complicates the matter further. More thought and discussion is needed on this point. ­Thoughts on Optimisations.­ I have been working on some server optimisations recently, and i aside from the things you can do with a LAMP stack, i have been giving thought to what i can do to further optimise the codebase. Some of the thoughts i was having was: ­
  • Cache the lexeme tables pre-process work from the BBCode-engine in Symfony2's cache to lighten the load per request if the pre-processing is not needed to be done per request­
  • ­
  • Cache the output of the BBCode engine and store it in a new column to the database, so that we don't have to render the BB tags per request, we can instead just parse the content during posting/editing and pull the pre-compiled html from the db when viewing posts­
  • ­
  • Cache twig globals instead of just using the bundles boot method, which at the moment is the only means i know of to allow configurations to be read from the bundles config and copied into the twig globals. This might not be possible for a while unless i can work something out.­
  • ­


There are more minor fixes and changes to the site overall than i can care to remember or list, though you are always welcome to check the github and look through the commit log if you want to know more. This early deployment is a bit of a beta test as i have not yet added the version tags, so if you wish to use this version on your own site then you will need to use master. Which is fine if your still in early dev, however if your wanting to use it on production then you i would recommend waiting a little while until the next stable tags are made available. ­


Please contribute by either posting here on the forum your questions, feedback or issues. Also please use the issue tracker on github if you know how to use it, if not then post bug information here and i will file a ticket for you. ­CodeConsortium on github­ ­CodeConsortium on twitter­ ­My personal blog­ Use ­Trello­ to track the progress and todo lists for the various project bundles.­