I have optimized the grid to load much faster and it is reaching a very usable speed.  The problem is I had to remove some good data from the API call in order to get this speed.  For testing, it is working fine.  I will now try to optimize the Grid class further so that I wont have to use such large API calls to retrieve all data and will still be able to display enough relevant information.  This data flaw is only related to the nodes list instantiation of the grid for choosing nodes for association for slices, other uses of the grid should be fine.

Adding sorting is still going to be a problem due to how slow our current sorting function are, relating to the dictionaries returned by API calls.  This is currently the main limitation of the grid and will apply to all future uses of grids anywhere in the GUI.  I will work on making stronger sorting functions in order to optimize this limitation.

Need to get it to display in drupal GUI, currently the way drupal loads a page is by loading it all into a buffer then displaying it all at once rather than sequentially, this is preventing it from displaying withing drupal.  Ill need to look into how to get the Grid or any ajax functionality to load when the page is printed from a buffer. 

Sorting on column heads is also a work in progress.  Editing on the fly is also being tested and has a few bugs.  This will likely be how many pages will work now, allowing editing on the fly with no page reloads.  The slowness in teh grid is only tied to large API calls, with smaller filtered calls the AJAX loads are barely noticeable.

The grid has been built using the API instead of direct DB queries.  The porting from DB to API was much more complex than predicted, but a few more things were learned in the process.  The grid is a LOT slower now that it uses the API calls, it was almost instant using the DB, but now there is a 3-5 second delay in refreshing the data as you paginate.  More optimization of the PHP and JScript code will have to be made in order to speed this up.  Also, the grid does not display within drupals GUI as of now.

 Current Bugs:

• long load time 
• diplay in drupal GUI <

CREATE TABLE hostnames (
  hostname varchar(255),
  ip int unsigned,
  PRIMARY KEY (hostname)
) PACK_KEYS=1;

--
-- Table of flows tables
--
CREATE TABLE flow_tables (
  table_name varchar(64),	-- name of the table
  src_ip int unsigned,		-- IP source address
  slice varchar(32),		-- slice that sent these flows
  start_time datetime,		-- start time of the earliest flow in the table
  end_time datetime,		-- end time of the latest flow in the table
  flows int unsigned,		-- total number of flows in this table
  packets bigint unsigned,	-- total number of packets sent in all flows in this table 

The database is horribly inefficient.  Looking into a new schema that will use the same data inputs and actually use relations within the database to link the data.  Currently, a table is created for each flow and then an entry is made into a main table which keeps track of all the flow tables.  In order to query this data, you have to run 2 queries: 1 to the main TOC table to find the name of the flow table, then another to query that table and get the flow data.  This is a perfect example of how NOT to use a relational database.  Looking into the most effecient storage model and nomalization for this system.  2NF should be fine due to the nature of the data.

PlanetFlow

Works in progress:

• DB normalization/optimization
• GUI update
• Usability analysis
• Kernal update
• New data storage and retrieval

After discussions with Stephen about our functional web code, we are beginning an effort to write code in a much more modular easy to follow style, much more like the current API sturcture.  This means that all future updates and changes to the web will be geared toward simplifying the code we write by using more of both drupals features as well as the API itself.  Building more Classes of objects rather than creating each page with a seperate script. 

Stephen has also been working on rebuilding our functional web pages into drupal nodes/pages.  He talks about it here.  This is a cool idea because it will allow drupal to cache and search through our pages as well as using many other features within durpal that we have been unable to tap into due to the split nature of the website.

I have merged all the functional code of both repositories and am going to deploy it on the test server to start bug tracking.  Functional code is basically all the code that site sin the /planetlab/ dir of the web root.  This is all the basic code needed to interface with the API outside of drupal.  Any various configurations and drupal settings, we felt, were not necessary parts of the merge because one of the main strengths of using drupal is to allow each PLC to maintain their own GUI appearances and styles.  Once i start debugging and testing I'll post the big changes in the merge as I get to them and talk about the differences and reason for style of merge. 

Installation of both MyPLCs has begun and setup and syncronizing the DBs is underway. We will be spending the next few days documenting both sides of the code. We will stay in close contact with 1 Lab for the duration of documentation in case there are any questions from either team. The goal is to come up with a full diff list between the two code bases with logica descriptions of what the diff is and why it is good or bad so that both teams can discuss and come to a consensus of which should go and which should stay when we merge the two codebases.

There will be some things that should remaind different as there are some fundamental differences on how both teams do business, but for the most part the codebases should match as much as possible during the development process so that both teams can learn from each others successes and failures.

 Merging 1Lab and PLC web GUIs

 Works in progress:

• Installing both MyPLCs
• Documenting both GUIs
• Forming a diff document of the two
• Discussing the good and the bad
• Merging the two code bases