cooldude234,

Whenever I have tried to determine whether a line segment (like an edge) intersects a polygon, I have done it in two steps:

1) determine if and where the line segment intersects the plane of the polygon

2) determine if that intersection is actually inside the polygon

It is unclear from your code segment what Ray1 and Ray2 actually are, but if they are points that define a line segment, then this is how I would solve this problem:

facenormal=normalize(crossproduct(vert3-vert2,vert2-vert1)) //get normal of polygon plane

rayvector= ray2 - ray1 //get vector of ray line segment

ray_plane_angle=dot(facenormal,rayvector) //actually the arccos of the angle between

if ray_plane_angle != 0 //if zero, the two are parallel

ray_plane_intersection_point = ray2 + ((dot(vert1 - ray2,facenormal) / ray_plane_angle) * rayvector

//above, calculate the intersection point of vector of the line segment and plane

//next add angles of intersection point to two of the vertices at a time.

//if the angles add to 360 deg (or 2*pi rad) then inside, otherwise the intersection point is outside

total_angles= acos(dot(normalize(vert1-ray_plane_intersection_point),normalize(vert2-ray_plane_intersection_point)))

+acos(dot(normalize(vert2-ray_plane_intersection_point),normalize(vert3-ray_plane_intersection_point)))

+acos(dot(normalize(vert3-ray_plane_intersection_point),normalize(vert1-ray_plane_intersection_point)))

if (abs(total_angles - 2*pi) < 0.001) contact_point=ray_plane_intersection_point

If you are actually dealing with a line segment, rather than an endless line, then you need to check whether the intersection point is actually between ray2 and ray1:

if length(ray2-ray_plane_intersection_point)+length(ray1-ray_plane_intersection_point) > length(ray2-ray1)

//intersection point is not between ray2 and ray1

I wrote the code in pseudo code rather than a particular language.

I hope that helps.

NickE