$7.49 .com domain with free InstantPage Website Builder$1.99 Web Hosting   捷飞网络官方淘宝店   Godaddy 优惠码
返回列表 发帖
分享到:




[PHP 语言] 分享php删除文本文件中重复行的脚本

  1. /**
  2. * RemoveDuplicatedLines
  3. * This function removes all duplicated lines of the given text file.
  4. *
  5. * @param     string
  6. * @param     bool
  7. * @return    string
  8. */
  9. function RemoveDuplicatedLines($Filepath, $IgnoreCase=false, $NewLine="\n"){

  10.     if (!file_exists($Filepath)){
  11.         $ErrorMsg  = 'RemoveDuplicatedLines error: ';
  12.         $ErrorMsg .= 'The given file ' . $Filepath . ' does not exist!';
  13.         die($ErrorMsg);
  14.     }

  15.     $Content = file_get_contents($Filepath);

  16.     $Content = RemoveDuplicatedLinesByString($Content, $IgnoreCase, $NewLine);

  17.     // Is the file writeable?
  18.     if (!is_writeable($Filepath)){
  19.         $ErrorMsg  = 'RemoveDuplicatedLines error: ';
  20.         $ErrorMsg .= 'The given file ' . $Filepath . ' is not writeable!';   
  21.         die($ErrorMsg);
  22.     }

  23.     // Write the new file
  24.     $FileResource = fopen($Filepath, 'w+');      
  25.     fwrite($FileResource, $Content);        
  26.     fclose($FileResource);   
  27. }

  28. /**
  29. * RemoveDuplicatedLinesByString
  30. * This function removes all duplicated lines of the given string.
  31. *
  32. * @param     string
  33. * @param     bool
  34. * @return    string
  35. */
  36. function RemoveDuplicatedLinesByString($Lines, $IgnoreCase=false, $NewLine="\n"){

  37.     if (is_array($Lines))
  38.         $Lines = implode($NewLine, $Lines);

  39.     $Lines = explode($NewLine, $Lines);

  40.     $LineArray = array();

  41.     $Duplicates = 0;

  42.     // Go trough all lines of the given file
  43.     for ($Line=0; $Line < count($Lines); $Line++){

  44.         // Trim whitespace for the current line
  45.         $CurrentLine = trim($Lines[$Line]);

  46.         // Skip empty lines
  47.         if ($CurrentLine == '')
  48.             continue;

  49.         // Use the line contents as array key
  50.         $LineKey = $CurrentLine;

  51.         if ($IgnoreCase)
  52.             $LineKey = strtolower($LineKey);

  53.         // Check if the array key already exists,
  54.         // if not add it otherwise increase the counter
  55.         if (!isset($LineArray[$LineKey]))
  56.             $LineArray[$LineKey] = $CurrentLine;        
  57.         else               
  58.             $Duplicates++;
  59.     }

  60.     // Sort the array
  61.     asort($LineArray);

  62.     // Return how many lines got removed
  63.     return implode($NewLine, array_values($LineArray));   
  64. }

  65. //使用范例:
  66. // Example 1
  67. // Removes all duplicated lines of the file definied in the first parameter.
  68. $RemovedLinesCount = RemoveDuplicatedLines('test.txt');
  69. print "Removed $RemovedLinesCount duplicate lines from the test.txt file.";

  70. // Example 2 (Ignore case)
  71. // Same as above, just ignores the line case.
  72. RemoveDuplicatedLines('test.txt', true);

  73. // Example 3 (Custom new line character)
  74. // By using the 3rd parameter you can define which character
  75. // should be used as new line indicator. In this case
  76. // the example file looks like 'foo;bar;foo;foo' and will
  77. // be replaced with 'foo;bar'
  78. RemoveDuplicatedLines('test.txt', false, ';');
复制代码
& z8 u  |# F1 W+ ^! B$ @4 b

" t  O  l8 ~# e参考: http://outofmemory.cn/code-snippet/6027/php-delete-text-file-repeat-row

美国VPS、域名代购:http://tu8l.taobao.com

PHP+MYSQL 两行Query搞定删除重复数据

  1. $db=mysql_connect('localhost','xxx','xxx');
  2. //text为可能存在重复的字段名,先用count子句来把TEXT重复的记录的ID拎出来,不全拎,节省资源~

  3. mysql_select_db('vbnew');
  4. $sql="SELECT id, text, count( text )
  5. FROM `dic`
  6. GROUP BY text
  7. HAVING count( text ) >1";
  8. $result=mysql_query($sql)or die("Invalid query: " . mysql_error());
  9. $i=1;
  10. //把S出来的记录的ID做成Array

  11. while($ids=mysql_fetch_array($result))
  12. {
  13. $id[$i]=$ids[0];
  14. echo $ids[0];
  15. echo "id为".$id[$i];
  16. echo "
  17. ";
  18. $i++;
  19. }
  20. //开始爽删~

  21. foreach($id as $a=>$b)
  22. {
  23. $sql="delete from dic where id=".$b."";
  24. mysql_query($sql);
  25. echo "成功删除1行,id为".$b;
  26. echo "
  27. ";
  28. }
  29. ?>
复制代码

" v0 W) z# ~+ n% \* `' u6 B  Q$ w
//做个小结 也算是个疑问吧 就是本来想着这种方法应该是把所有重复的记录都删掉,没想到却恰好留一条,也算歪打正着吧~
, I1 V8 a6 G) c, i
, e; B6 W" j; t7 e1 o9 ~! @% v. M# }
参考: http://www.coder4.com/archives/76

美国VPS、域名代购:http://tu8l.taobao.com

TOP

返回列表
Namecheap
Namecheap.com - Cheap domain name registration, renewal and transfers - Free SSL Certificates - Web Hosting
互联网安全