PDA

View Full Version : PHP and Hot Folders. Is this possible?


smitho
02-21-2008, 04:44 AM
Not having much luck finding information on a hot folder process in PHP.

Would it be possible to have a web page poll a folder and when it finds new files upload and process them? Rather that the user having to click the browse for the file approach.

davidj
02-21-2008, 06:02 AM
you could write a script which will run on a cron and look in a Dir for new files

once detected, process the files and then move them out into an archive/processed location

smitho
02-21-2008, 09:12 AM
Sorry davidj,

could you explain "cron". I gather from your responce php is capable of polling a folder and uploading the files. I'll do a searh on cron and see what I get.

Cheers.

davidj
02-21-2008, 09:44 AM
its a linux program

cron or crontab allows you to run a script at an interval of your choosing

its like a schedule in windows

davidj
02-21-2008, 09:45 AM
PHP can read, write and manipulate files within any dir as long as it has the rights to do so

davidj
02-21-2008, 09:48 AM
let me know if you want help with this

smitho
02-25-2008, 01:52 AM
After spending 2 days rebuilding my PC I'm ready to get back on this topic.

This is the process I have down so far.

A user can copy a number of files into a folder (in this example I'm using .jpg).

I have a page that polls that folder and determines that if there are files in that folder do the first step:

1. copy files to new folder but rename them to a set number instead of their current filename.

To get a set number I've created a text file with specific numbers. What I'm having a problem with is getting the process to poll through the numbers and rename the files with one of the numbers.

The numbers file (number.txt) looks like this:
100001,100002,100003,100004,100005This is what I'm working on.


<?php
if ($handle = opendir('./images')) {
while (false !== ($file = readdir($handle))) {
if ($file != "." && $file != ".." && $file != ".DS_Store"){
echo "$file<br>";

$myFile = "numbers.txt";
$fh = fopen($myFile, 'r');
$theData = fread($fh, filesize($myFile));
fclose($fh);
$anumber = explode(",", $theData);

$number= $anumber[0]++;

$tocopy = "images/$file";
$justname = substr($file, 0, -4);

$copyfile = "process/$number";

if (!copy($tocopy, $copyfile)) {
echo "failed to copy $file...\n";


}
}
}
closedir($handle);
}
?>
I have 4 jpg images in the /image folder called image_1.jpg to image_4.jpg. When I run the code only 100001 is created in the process folder. Is there a way to I could read the numbers file and poll through the list.

So the end result should be:

image_1.jpg => 100001
image_2.jpg => 100002
image_3.jpg => 100003
image_4.jpg => 100004

smitho
02-25-2008, 02:53 AM
Made a couple of changes but still what I'm after.

I've added:



$i=0;
if ($handle = opendir('./images')) {...................

//Then changed $anumber[0] to

$i++;
$number= $anumber[$i];

The 4 jpg files are being renamed but they are not starting at 100001 they're stating at 100002. Is there a way to get the number to start at 100001?

Also as there are only 5 numbers in the list could I set it up to only take the first 5 files in the folder then once completed grab the next five?

davidj
02-25-2008, 05:39 AM
try changing this...

$i=0;

to this


$i=-1;

smitho
02-25-2008, 08:19 AM
Is it possible to copy a file to an IP address? I'd like to try and copy/move the files to a server once the process is complete.

Also I would like to try and make the process use all the numbers consecutively. So if a user has only drop in 2 files in one session the numbers 100001 and 100002 would be used. The next time they drop files the process would start at 100003. Would using sessions be the way to go to try and remeber what the last number uses would be?

davidj
02-25-2008, 08:23 AM
Is it possible to copy a file to an IP address? I'd like to try and copy/move the files to a server once the process is complete.

You could write an bash script in linux which would FTP the contents of a dir. This would have to be run from a cron

Also I would like to try and make the process use all the numbers consecutively. So if a user has only drop in 2 files in one session the numbers 100001 and 100002 would be used. The next time they drop files the process would start at 100003. Would using sessions be the way to go to try and remeber what the last number uses would be?

you would have to store the last numeric in a db and access that when processing files incrementing it by 1

davidj
02-25-2008, 08:27 AM
you could run the above from a user event rather than a cron but to connect to another machine via an action performed by a user is not reliable. I would perform this transfer after hours, once a day, possibly at midnight when data traffic is low.

smitho
02-25-2008, 08:53 AM
Thanks davidj,

The next part of the process is to create a dynamic table of the files that have been dropped in and renamed so this should work well with getting the last number used from the database.

All the dev work I'm doing is on PC and Mac systems. I did a cron search on google and found there are similar process for PC and Mac so I might try this to see if I can do it. The other issue I thougt about was all the links I have in the database for the files would need to be replaced/changed. The reason I would like to be able to do this is for archive purposes.

I'll start with the database for the files being dropped into the folder and renamed.

Just talking about databases I've completed the first part of the system I'm working on and I didn't ask what the best way to setup the database was. But what I did was create a number of tables (5) for each site that would be uploading files for distribution. I though this would be better rather than keeping them all in one database. I started with one database but changed it to reflect each site. Was that a good or bad move?

davidj
02-25-2008, 08:56 AM
remember to avoid replication of data. If any db gets out of sync its a nightmare to admin

one db is the way to go in my opinion

smitho
02-26-2008, 09:38 AM
I've got the process working so that when a user drops a number of files in the folder the server will read the folder copy the file to the process folder and rename the files with one of the stored numbers, Even got it to check the last used number and then cycle through the list. I've also set it to insert the filename and number into a table.



/////////////////////////////////////
$query = sprintf("SELECT * FROM list ORDER BY id DESC LIMIT 1");
$result = mysql_query($query);
$rowNumber = mysql_fetch_array($result);
//$totalNumber = mysql_num_rows($result);
/////////////////////////////////////
$usednumber= $rowNumber['number'];
//echo "Number".$rowNumber['number']."<br>";

/////////////////////////////////////
$query2 = sprintf("SELECT * FROM numbers WHERE num='$usednumber'");
$result2 = mysql_query($query2);
$rowNumberid = mysql_fetch_array($result2);
/////////////////////////////////////
//echo "id".$rowNumberid['id']."<br>";
$i=$rowNumberid['id'];

///////////// First Folder //////////////////

if ($handle = opendir('./images')) {
while (false !== ($file = readdir($handle))){
if ($file != "." && $file != ".." && $file != ".DS_Store"){
//echo "$file<br>";
chmod("images/$file", 0777);
chown("images/", "www");

if($i>=30){
$i=1;
}else{
$i++;
}
//echo "i".$i."<br>";

/////////////////////////////////////
$query = sprintf("SELECT * FROM numbers WHERE id='$i'");
$result = mysql_query($query);
$rowNum = mysql_fetch_array($result);
/////////////////////////////////////

$number= $rowNum['num'];

$tocopy = "images/$file";
//$justname = substr($file, 0, -4);

$copyfile = "process/$number";

//echo "$number<br>";

$query = sprintf("INSERT INTO list (date, provider, filename, number) VALUES (now(), 'Images', '$file', '$number')");
mysql_query($query)or die(mysql_error);

if (!copy($tocopy, $copyfile)) {
echo "failed to copy $file...\n";
}
}
}
closedir($handle);
} I have a slight problem when I test this on the Mac platform. Not all but some files do not have the correct rights, so the script falls over when it tries to rename the file and copy it to the processed folder. This does not happen on the windows platform when I test the same files. I tried using chmod and chown but it did not seem to make a difference. Does chmod and chown only work on files that have been uploaded into PHP?

Not sure how I can give the server access to the files. I've changed the privileges on the folder but that does not seem to matter when you dump the files in. If I apply the privileges to the folder a second time with the files in the folder the script works fine.

davidj
02-26-2008, 09:43 AM
anything uploaded by your php app should be owned by apache so you shouldnt have a problem

can you not set the rights of the process dir manually

davidj
02-26-2008, 09:58 AM
can i suggest a method of error display and handling

set all your errors as constants in a file which you would include. In the file called errors.php set them like so...


define('ERR001','message here');
define('ERR002','another message here');

The first argument is the handle and the second is the message.

To display the error just assign the handle to a var like so and echo the $error var where you want


$error = ERR001; // << note no quotes


this is a more professional way to store your messages in one page. Like most pro apps you can use the handle for logging the error rather than storing the message string. The handles and errors will be documented in your functional spec

as you can see i am using this naming convention 'ERR001' (xxxnnn) where nnn is a numeric range so i could allocate the following range...

001 > 100 as db errors
110 > 200 as app errors
500 > 999 could be user information and warnings like "your query returned empty results "

smitho
02-26-2008, 11:31 AM
Thanks davidj,

When you say anything uploaded by your php app should be owned by apache so you shouldnt have a problem I am not uploading these files with the upload process I used before. What I was looking for was a way for a person to grab a batch of files 5, 10, 15 or 20 at a time and drop them into a folder. The folder would mirror/ftp the files to the folder on the web server (still looking into this cron approach).

At the moment to get the code to work I'm dumping a number of files in the images folder and then running the php page that will process the files in the folder. So technically the files are not uploaded by the php app. In saying that the process does work just that the Mac folder rights process is not working the way I want.

Can I suggest a vid tut on setting up a better way to return errors for your code. There's a couple of places in my code I need to include what you have suggested.

davidj
02-26-2008, 12:38 PM
its a strange one

i don't know why your mac is having problems

are the rights set correctly in the process/parent Dir.

regarding a vid on error handling

the procedure is a simple one explained above. There is not enough content to warrant a vid