c# - Parallel.Foreach is to0 slow. How can I optimize my code -


i have flollowing code takes more 7 minutes complete, when use parallel.foreach. list "final_products" iterate on contains 7000 products.

 public void generatetreefromallfinalproducts()     {         xmlserializer serializer = new xmlserializer(typeof(imagefeature<float>[]));         dstableadapters.products_uniquetableadapter pft = new dstableadapters.products_uniquetableadapter();         dstableadapters.products_unique_surftableadapter pus = new dstableadapters.products_unique_surftableadapter();         ds.products_uniquedatatable final_products = pft.getdata();          stopwatch stopwatch = new stopwatch();         stopwatch.start();          parallel.foreach(final_products.asparallel(), row =>         {                              //get surf data images found similar image             types.products_unique_surfrow surfdata = getdatabyuniqueproductid(row.id);              imagefeature<float>[] row_features = (imagefeature<float>[])serializer.deserialize(new stringreader(decompress(surfdata.surf)));             if (row_features != null)                 flann.addsurfdescriptors(row_features, row.id);                                });          stopwatch.stop();         console.writeline("time elapsed: {0}", stopwatch.elapsed);     } 

is normalt takes long , if not how can optimize code?

getdatabyuniqueproductid(row.id) call database returns 1 single row.

private static types.products_unique_surfrow getdatabyuniqueproductid(int rowid)     {         types.products_unique_surfrow ret = new types.products_unique_surfrow();          string sqltext = "set rowcount 1 select surf products_unique_surf unique_product_id =" + rowid;          using (sqlconnection myconn = new sqlconnection(global::scbot.properties.settings.default.dataconnectionstring))         {             myconn.open();              sqlcommand cmd = new sqlcommand(sqltext, myconn);             try             {                 cmd.commandtype = commandtype.text;                  sqldatareader reader = cmd.executereader();                 while (reader.read())                 {                     types.products_unique_surfrow row = new types.products_unique_surfrow();                     row.surf = convert.tostring(reader["surf"]);                      ret = row;                 }             }             catch (exception e)             {                 messagebox.show(e.tostring());             }         }         return ret;     } 

my initial code following

 public void generatetreefromallfinalproducts()     {         xmlserializer serializer = new xmlserializer(typeof(imagefeature<float>[]));         dstableadapters.products_uniquetableadapter pft = new dstableadapters.products_uniquetableadapter();         dstableadapters.products_unique_surftableadapter pus = new dstableadapters.products_unique_surftableadapter();         ds.products_uniquedatatable final_products = pft.getdata();          foreach (ds.products_uniquerow row in final_products)         {             //get surf data images found similar image             list<ds.products_unique_surfrow> surfdata = pus.getdatabyuniqueproductid(row.id).tolist();              foreach (ds.products_unique_surfrow data in surfdata)             {                 imagefeature<float>[] row_features = (imagefeature<float>[])serializer.deserialize(new stringreader(decompress(data.surf)));                 flann.addsurfdescriptors(row_features, row.id);             }         }     } 

but slow, why tried parallel.foreach

you should measure time inner code takes. parallel won't speed-up inner code. instead of nesting methods decompress/deserialize, measure them separately.

after edit:

each thread create new connection. think great example how not use parallel speeding up. changing algoritme more. because querying single rows on different connections/threads wll cost more time query them (or selection) , single threaded foreach loop them.

my view:

i collect id's of final_products enemration , buildup string using stringbuilder.

stringbuilder sb = new stringbuiler(); bool isfirst = true;  sb.append("("); foreach(var prod in final_products) {     if(isfirst)         isfirst = false;     else         sb.append(", ");      sb.append(prod.id); } sb.append(")");  string query = "select surf products_unique_surf id in "+sb.tostring();  // execute query  // foreach row, decompress, deserialize etc... 

Comments

Popular posts from this blog

matlab - Deleting rows with specific rules -

php - MySQLi multi_query results for later use -