I am trying to grab the description of object made in my workshop which where sold thru ebay during the last three months because I lost my archives.
I have firstNums and lastNums to search for so this is not the problem.
I wrote this script:
set {numB, num0} to {"2200", 42264273} -- 220042264273 is a GERBINO
set {numB, num0} to {"2200", 42264271}
set kWord to "gerbino"
tell application "Safari" to activate
repeat with i from 0 to 3
set numF to num0 + i
if (my cherche(numB, numF, kWord)) then ¬
display dialog "Found " & kWord & " in «" & numB & numF & "»"
end repeat
on cherche(B, F, t)
local k, nw1, ve1
tell application "System Events" to tell application process "Safari"
set nw1 to get name of window 1
(*
tell window 1 to set url1 to ¬
get value of text field 1 of splitter group 1 of group 4 of tool bar 1
*)
tell window 1 to tell group 2 to tell scroll area 1 to tell UI element 1
try
set k to 6
get name of button 1 of group k
on error
set k to 8
end try
tell group k
set value of text field 1 to (B & F)
delay 0.1
click button 1
end tell
end tell -- window 1
end tell --process then System Events
tell application "System Events" to tell application process "Safari"
repeat
set nw2 to get name of window 1
delay 0.1
if nw2 is not nw1 then exit repeat
end repeat
tell window 1
tell group 2 to tell scroll area 1 to ¬
set ve1 to get value of UI element 1
end tell -- window 1
end tell --process then System Events
return (nw2 & ve1) contains t
end cherche
Which, if it was working correctly would discover that object #220042264273 is a GERBINO one.
Alas when the 220042264273 num is “entered” it seems that the search process doesn’t run.
I hope that one of you will be able to discover what is wrong.
Yvan KOENIG (from FRANCE vendredi 3 novembre 2006 10:51:47)
I’m afraid I can’t help with the solution as my Tiger machine isn’t connected to the Internet. But I can tell you “ and in particular anyone else who may want to help you “ that the numbers of the groups in a Safari window depend on how many bars (Address bar, Bookmarks bar, Status bar) you have displayed. You’re apparently trying to script the displayed Web page, which seems always to be the last group, group -1 of window 1.
Thereafter, the UI elements of UI element 1 of scroll area 1 depend on the page being displayed, so you’ll need to provide the URL for that.
I admit I haven’t tried to understand your UI scripting and maybe I totally misunderstood your problem - forgive me if so … but as far as I understood you problem, I thought it could more reliable/easier if you’d use other techniques for your search:
JavaScript for filling in the search form and starting the search:
set searchString to "220042264273"
tell application "Safari"
tell document 1
do JavaScript "document.forms[0].elements[1].value = " & searchString in it
do JavaScript "document.forms[0].elements[2].click()" in it
end tell
end tell
and parsing the page source for the ebay item description:
set {od, text item delimiters} to {text item delimiters, "ebay: "}
tell application "Safari" to tell document 1 to set desc1 to text item 2 of (paragraph 3 of (get source))
set text item delimiters to "</title>"
set desc to text item 1 of desc1
set text item delimiters to od
get desc
1 - I never used javascript so, I will look at your proposal but I hope that I may do the trick with “standard” AppleScript.
2 - Nigel GARVEY put me on a “good” track and I wrote:
set {numB, num0} to {"2200", 42264273} -- 220042264273 is a GERBINO
set {numB, num0} to {"2200", 42264260}
set kWord to "gerbino"
tell application "Safari" to activate
(*
http://cgi.ebay.fr/ws/eBayISAPI.dll?ViewItem&item=220042264273&ru=http%3A%2F%2Fsearch.ebay.fr%3A80%2Fsearch%2Fsearch.dll%3Ffrom%3DR40%26satitle%3D220042264273%26fvi%3D1
*)
set maliste to {}
repeat with i from 1 to 20
set numF to num0 + i
if (my cherche(numB, numF, kWord)) then
copy (numB & numF & return) to end of maliste
set the clipboard to maliste as text
end if
end repeat
on cherche(B, F, t)
local nw1, ve1
tell application "Safari" to open location ("http://cgi.ebay.fr/ws/eBayISAPI.dll?ViewItem&item=" & B & F)
delay 3
tell application "System Events" to tell application process "Safari"
set nw1 to get name of window 1
tell window 1 to tell group 2 to tell scroll area 1 to ¬
set ve1 to get value of UI element 1
end tell -- window 1 then process then System Events
tell application "Safari" to close window 1
return (nw1 & ve1) contains t
end cherche
It works.
The only remaining problem is: how may I test if the open location call is finished ?
At this time I put a delay but I’m sure that most of the time, it is too long and that some of these days, it will be too short.
I wrote “some of these days” because the number of ebay’s objects is really huge.
Yvan KOENIG (from FRANCE vendredi 3 novembre 2006 17:37:42)
this is a subroutine using javascript to do the job
if page_loaded(20) is false then error nummer - 128 -- or whatever
on page_loaded(timeout_value)
delay 2
repeat with i from 1 to the timeout_value
tell application "Safari"
if (do JavaScript "document.readyState" in document 1) is "complete" then
return true
else if i is the timeout_value then
return false
else
delay 1
end if
end tell
end repeat
return false
end page_loaded
You might also consider a curl-based routine, Yvan:
set search_site to "http://cgi.ebay.fr/ws/eBayISAPI.dll?ViewItem&item="
set numB to "2200"
set num0 to 42264260
set kWord to "gerbino"
set maliste to {}
set search_base to search_site & numB
repeat with i from num0 to num0 + 20
if kWord is in (do shell script "curl " & quoted form of (search_base & i)) then set maliste's end to numB & i
end repeat
maliste --> {"220042264273"}
Now, I have a script which works.
It scans about 1000 objects in an hour but I assumes that the lack of speed is linked to my internet connection.
Here is the code
set {numB, num0} to {"2200", 24300036} -- 220042264273 is a GERBINO
set kWord to "gerbino"
tell application "Safari" to activate
set maliste to {}
repeat with i from 1 to 2000
set numF to num0 + i
set _ to my cherche(numB, numF, kWord)
if _ = "abandon" then
copy ("abandon " & numB & numF & return) to end of maliste
set the clipboard to maliste as text
else if _ then
copy (numB & numF & return) to end of maliste
set the clipboard to maliste as text
end if
end repeat
on cherche(B, F, t)
local ok, nw1, ve1
tell application "Safari" to open location ("http://cgi.ebay.fr/ws/eBayISAPI.dll?ViewItem&item=" & B & F)
set ok to my page_loaded(20)
if ok then
tell application "System Events" to tell application process "Safari"
set nw1 to get name of window 1
tell window 1 to tell group 2 to tell scroll area 1 to ¬
set ve1 to get value of UI element 1
end tell -- window 1 then process then System Events
tell application "Safari" to close window 1
else
try
tell application "Safari" to close window 1
end try
end if
if ok then
return (nw1 & ve1) contains t
else
return "abandon"
end if
end cherche
on page_loaded(timeout_value)
local i
delay 1
repeat with i from 1 to the timeout_value
tell application "Safari"
if (do JavaScript "document.readyState" in document 1) is "complete" then
return true
else if i is the timeout_value then
return false
else
delay 1
end if
end tell
end repeat
return false
end page_loaded
Yvan KOENIG (from FRANCE vendredi 3 novembre 2006 22:35:48)
Connection speed is only part of the equation, Yvan.
Apart from the downloading of a web page, one of the main bottlenecks is the time it takes a browser like Safari to display HTML code as formatted text and graphics. (That’s why I suggested the curl alternative, which generally performs about 7 or 8 times faster here.)
do shell script "curl http://http://bbs.applescript.net/post.php?tid=19043"
is the source code of this thread.
I modified your code to grab the pages with curl. Because there are a few pages which are redirected to another location
it’s necessary to use the -L flag. The script runs much faster than the “open location” version, the disadvantage is having
no display of progress
set {numB, num0} to {"2200", 24300035} -- 220042264273 is a GERBINO
set kWord to "gerbino"
set maliste to {}
repeat with i from 1 to 2000
set numF to num0 + i
set theURL to quoted form of ("http://cgi.ebay.fr/ws/eBayISAPI.dll?ViewItem&item=" & numB & numF) & " | grep '<title>'"
set GrabSource to do shell script "curl -L " & theURL -- the L-flag enables curl to work also on redirected pages
set oldDelims to AppleScript's text item delimiters
set AppleScript's text item delimiters to {"<title>"}
set GrabSource to text item 2 of GrabSource
set AppleScript's text item delimiters to {"</title>"}
set GrabSource to text item 1 of GrabSource
set AppleScript's text item delimiters to oldDelims
if kWord is in GrabSource then
copy (numB & numF & return) to end of maliste
set the clipboard to maliste as text
else
copy ("abandon " & numB & numF & return) to end of maliste
set the clipboard to maliste as text
end if
end repeat
I looked at the curl’s man and was afraid by the numerous options.
With the given code it appears easier.
1 - when running the kai’s one I got an error message so I decided to introduce a try / end try block which does the trick.
2 - the first stephanK’s code gave an error on my machine:
the log report displaid:
tell current application
do shell script “curl http://http://bbs.applescript.net/post.php?tid=19043”
"
curl: (6) Couldn’t resolve host ‘http:’"
3 - the commented second code was interesting and was uselessly complicated
In my own one, I searched for the kWord in the title AND in the page’s body so there is no need to use grep and the TIDs.
4 - the message beginning with "abandon " was not stored when the kWord is missing but when something failed in the grab infos process.
So I modified the code adding an option to the call to curl.
set {numB, num0} to {"2200", 24300035} -- 220042264273 is a GERBINO
set kWord to "gerbino"
set maliste to {}
repeat with numF from num0 to num0 + 2000
try
set GrabSource to do shell script "curl -L --connect-timeout 120 " & (quoted form of ("http://cgi.ebay.fr/ws/eBayISAPI.dll?ViewItem&item=" & numB & numF)) -- the L-flag enables curl to work also on redirected pages
if kWord is in GrabSource then
copy (numB & numF & return) to end of maliste
end if
on error
copy ("err_" & numB & numF & return) to end of maliste
end try
end repeat
set the clipboard to maliste as text
Yvan KOENIG (from FRANCE samedi 4 novembre 2006 19:36:36)
You can also use curl to download a sequential series of files in one hit. Since this method reduces server connects/handshakes, it can be a bit faster than making individual calls.
An added benefit is that, by opening the container folder in Finder, you can get some idea of download progress from the number of files that it contains. (The ability to see files being added in this way should also demonstrate the speed advantages of curl over a browser for this type of purpose.)
to search for t at s against p from b to e
tell application "Finder" to set f to (make folder with properties {name:"ebay " & p & space & b & "-" & e}) as Unicode text
do shell script "curl -L " & quoted form of (s & p & "[" & b & "-" & e & "]") & " -o " & quoted form of (f's POSIX path & "#1")
set text item delimiters to return & f
set l to paragraphs of (f & (list folder f without invisibles))
repeat with i in l
if t is not in (read file i) then set i's contents to false
end repeat
set text item delimiters to ""
set l to (every Unicode text of l) as Unicode text
set text item delimiters to f
set l to rest of l's text items
set text item delimiters to {""}
l
end search
search for "gerbino" at "http://cgi.ebay.fr/ws/eBayISAPI.dll?ViewItem&item=" against "2200" from 42264261 to 42264300
--> {"42264273"}
I made some changes. I no longer build a list. I use the kai’s scheme and move the matching files in a fixed folder where I may retrieve them.
Doing that, a single click opens the ebay’s page where I may grab the useful infos for my database.
property fFound : missing value
to search for t at s against p from b to e
tell application "Finder" to set f to (make folder with properties {name:"ebay " & p & space & b & "-" & e}) as Unicode text
do shell script "curl -L " & quoted form of (s & p & "[" & b & "-" & e & "]") & " -o " & quoted form of (f's POSIX path & "#1")
set text item delimiters to return & f
set l to paragraphs of (f & (list folder f without invisibles))
repeat with i in l
if t is in (read file i) then tell application "Finder" to move item i to fFound
end repeat
tell application "Finder" to delete alias f
end search
set p2d to (path to desktop) as Unicode text
set nFound to "ebayFound_dnuoFyabe"
tell application "Finder"
if not (exists folder (p2d & nFound)) then make folder at folder p2d with properties {name:nFound}
set fFound to p2d & nFound & ":" as alias
end tell
set {step, beg, |first|} to {20, "2200", 42264270}
repeat with k from |first| to |first| + (50 * step) by step
search for "gerbino" at "http://cgi.ebay.fr/ws/eBayISAPI.dll?ViewItem&item=" against beg from k to (k + step - 1)
end repeat
Yvan KOENIG (from FRANCE dimanche 5 novembre 2006 10:45:25)
