php - Compare 200.000 texts from DB -


i have mysql db , solr instance, exact copy of mysql db.

we have lot of duplicated texts (or texts simular more 80%). texts needs bundled on. in php know there functions calculate simulairities.

my problem is follows:

how can smart (read: fast possible) compare every new text, existing text in db? dont think practice read 200.000 records db/solr compare 1 new item, , next thing, again.

you install mysql plugin implementing levenshtein distance algorithm mysql , let database comparison.


Comments

Post a Comment

Popular posts from this blog

matlab - Deleting rows with specific rules -

i got 20*3 cell array , need delete rows contains "137", "2" , "n:t" origin data: 't' '' '' 'np(*)' '' '' [ 137] '' '' [ 2] '' '' 'are' 'and' 'np(fcc_a1#1)' '' '' '1:t' [ 1200] [0.7052] '' [1.2051e+03] [0.7076] '' 'are' 'and' 'np(fcc_a1#3)' '' '' '2:t' [ 1200] [0.0673] '' [1.2051e+03] [0.0671] '' 'are' 'and' 'np(m23c6)' '' '' '3:t' [ 1200] [0.227...
Read more

jquery - How would i go about shortening this code? And to cancel the previous click on click of new section? -

my code below changes image people in team sector, can see theres alot of repeated code; there way shorten it? also currently, allows click on sections without turning off clicked one; assume toggle() function, couldn't work other way! still quite new jquery coding, grand!! cheers in advance! $("#team_client_services").click(function () { if ($(".team_client_services").hasclass("team_hide")) { $(".team_client_services").toggleclass("team_hide"); $(".team_client_services2").toggleclass("team_show"); } else { $(".team_client_services").toggleclass("team_hide"); $(".team_client_services2").toggleclass("team_show"); }; }); $("#team_management").click(function () { if ($(".team_management").hasclass("team_hide")) { $(".team_management").toggleclass("team_hide"); $(".team_management2...
Read more